Hi, I’m Shibata, manager of AnyMind Tech team and leading AnyChat product now. We released a product called AnyChat as AnyMind Group this month, and in this article, I’d like to give you an introduction to AnyChat and the reasons why we applied Scala, Akka etc on the backend.
What is AnyChat
For the background and aim of this product, please refer to the official press release, but AnyChat was developed as a Conversational Commerce platform. We approached it as creating a CRM tool, and aim to maximize customer LTV by recommending appropriate products and providing customer support that matches the brand image, based on user’s info such as user attributes and purchase history.
We designed it for expanding globally like other products of AnyMind Group, and first we chose LINE as the chat tool to connect. In the near future, we’ll also connect with multiple chat tools that are widely used in Asia. At the same time, we plan to integrate with multiple online stores such as Shopify.
The overview of AnyChat system
The main premise of the functions required of chat systems will be "whenever a user sends a message, the system can respond immediately". If I use the words from Reactive Manifesto, the system should be "Responsive", "Resilient", "Elastic".
I picked up the "Reactive Manifesto", we are designing AnyChat’s backend to reactive system as much as possible.
The overall picture of the system is as follows, so that each microservice can install the optimum technology and design.
As a whole system, we chose GCP as our main infrastructure, and each microservice is deployed on Kubernetes (GKE Autopilot). The reason why I chose Kubernetes is that it’s not only the de facto standard tool, but also it allows us to fine-tune autoscaling and rolling updates, and even auto-recovering pods within the cluster.
For the communication protocol between services we opted to use gRPC. This is also a common technology, we like it because gRPC supports streaming transmission and automatically generates client code. By the way, to provide AnyChat features to other AnyMind products, we use Cloud Endpoint to make the gRPC endpoint public. I will omit the details in this article, but if you are interested, please check the below article.
[Tech Blog] Publish gRPC API using Cloud Endpoints & GKE
We chose backend language as Scala. The reason is Akka Cluster is really suitable for this application, and we can write the domain logic concisely. The domain logic is written using ZIO. we used Cats first, but compared to the code that mixes conversion to Monad Transformer and Lift, but ZIO can express various expressions with one type and there are many utility methods, so we can write domain logic much more straightforward.
OK, now that I’ve introduced the overall architecture, I will explain each microservice one by one, what kind of design policy they have, and what kind of technologies are selected.
LINE Service
This service is responsible for all communication with LINE. When a user sends a friend request or message on LINE, a webhook will be kicked from LINE, then this system will subscribe and execute the logic.
We’re trying to make this system reactive system as mentioned before, so that it’ll respond immediately without missing data whenever the user takes action, and try returning the content that matches the attributes of each user.
Technology-wise, we use message-driven approach using Akka Cluster. We assign an actor to each conversation between LINE Bot and the user. This actor manages the state of each conversation consistently and efficiently, and each Actor is distributed within the cluster, so it can be scaled horizontally. The Actor persists events sent from LINE to Cassandra (AWS KeySpace) as directly as possible. After that, run subsequent processing such as responding to the user and saving to Cloud Spanner using Akka Projection. Spanner data is optimized for reading from management console, and I think it is a typical pattern of CQRS + ES.
Shop Service
This service is responsible for all interactions with the online stores.
Generally, there are Entities called Customer, Product, and Order on online stores, so this system will keep a copy of them. And also this system coordinates requests of making a purchase on behalf of the user, detects the event when a Customer update or Order occurs, and notifies the events to others (CRM Service as detailed below).
For example, on Shopify, you’ll need to have a read-only copy because of the limited use of the Admin API and the response speed. In addition, every store requires unique implementation such as authentication logic and access token management, and the format of Order might be different, so we have designed AnyChat to absorb such differences and provide as unified an API as possible.
Since the data of the external store is master and the data handled by this service can be regarded as a copy, non-functional requirements such as availability are not strict and there is not much technical difficulty.
CRM Service
This service is responsible for managing channels for each customer and for subsequent processing from requests and notified events from other microservices. For example, when a product is purchased from an online store, We’ll notify users of the details on LINE, or link LINE users with Shop Customers.
Since this service has the role of connecting LINE users and Shop Customers, this service knows both LINE Service and Shop Service, and needs to perform various processing, so the domain logic is more complicated than other microservices. And in some cases, rollback processing such as Saga pattern is required. However, non-functional requirements are not strict, and even if there is a problem, in most cases there is no problem if an error is returned to the client or if it can be solved by retry.
Console Service
This service is responsible for accepting requests from Frontend and authenticating users.
On Technical aspect, we use GraphQL as the communication protocol with Frontend, and use Caliban which is a ZIO-friendly GraphQL library. We are still developing this, but we are planning a function to receive user inquiries, update the screen in real time with GraphQL subscription.
We also use Auth0 for authentication, and we are considering to a Single-Sign On function together with other products of AnyMind Group in the future.
Current issues and what we want to do next
Non-blocking DB access
Currently, we are using Spanner and PostgreSQL as the DB, and apply ScalikeJDBC as the client library on both sides, but since JDBC is a blocking API, I would like to optimize it more. However, even if PostgreSQL is probably fine in that case, Spanner still has to investigate whether to use the official Async query builder or choose Postgres mode – we haven’t researched it yet.
Performance tuning of Akka’s Actor logic
To manage state in Actors, we have to process events synchronously. However, in building the logic of AnyChat’s automatic response we want to call some external APIs, such as Q&A responses in cooperation with machine learning API etc. and changing the response contents according to the inventory and delivery status of the product. I feel this is a little tough issue to design a high-performance logic without conflicting state updates.
Upgrade to Scala 3 and ZIO 2.0
I’m using Intellij IDEA as Scala IDE, and I feel that the support for Scala 3 is still insufficient, so we’re still using Scala 2, but I’d like to move to Scala 3 at some point. For example, if we use Opaque Type or Enum Type, you can write domain logic more intuitively and type-safely.
Also, since the ZIO 2 series has reached RC2, I also want to try upgrading it when it’s officially released. There seems to be no change that greatly changes the usability, and I think that the merits are to make APIs more concise and improve the performance.
Expansion of marketing automation functions
AnyChat is a platform that holds an important communication path to customers, and by connecting to an online store, it is possible to analyze sales for each user in detail, so I think that there are huge potential to maximize customer LTV. Starting with how often you should send promotion notifications, product recommendations tailored to customers’ attributes, developing Q&A algorithms, and provide full functionalities of shopping on Chat UI, etc
Summary
I think the interesting points of AnyChat are both the technical difficulty of creating a chat system & business collaboration with marketers and customer support for the sake of maximizing customer LTV. Furthermore, In order to expand globally, it seems necessary to consider where should we put servers and how to deal with marketing strategies that differ from country to country.
we are hiring more engineers and product managers. If you are interested AnyChat, please select “product development” from the link below and see available positions.
https://anymindgroup.com/career/
We also have a blog written by engineers in AnyMind Group. If you’d like to read it, please click the links below:
- English: https://medium.com/anymind-group/tech-blog/home
- Japanese: https://qiita.com/organizations/anymindgroup
Finally, AnyMind is sponsoring ScalaMatsuri 2022. We will also open our virtual booth on the day, so please feel free to drop by if you have any questions including AnyChat development. (I will participate on the second day.)
