Hi, my name is Shibata and I’m manager of the engineering team at AnyMind Group. In this blog post, I’d like to introduce our AnyTag and AnyCreator products and how we have been developing them, along with our development system and technology stack.
■ About AnyTag and AnyCreator
First, let me explain what this product is all about. AnyTag and AnyCreator are platforms for influencer marketing, and is available across Asia. These platforms are one of the leading platforms in the region for influencer marketing, and we’re constantly innovating and improving them. We believe that there are two major types of users in influencer marketing, and we have divided our products accordingly to meet the needs of both sets of users.
The first is AnyTag (formerly known as CastingAsia), which is built for advertisers. Users can select and work with the best influencers that suit a brand, and also manage campaigns and track performance of campaigns. In the past year, we have also developed functionality to enable users to grow and analyze their own brand’s social media accounts.
The other platform is AnyCreator, which is built for influencers. The platform provides users with support for follower acquisition, content creation and monetization. We recently announced a business partnership with UUUM, and they are able to analyze data of their influencers and use this data for campaigns and support the launch of their influencers’ D2C brands.
Now that I’ve introduced the products, I’d like to share some of our key learnings and how we developed this product, which is widely used by our partners, including marketers, marketing agencies, influencer networks, talent agencies, and more.
■ Early-stage release
AnyTag and AnyCreator were initially released under the name CastingAsia. In the first version, Takemoto, the Managing Director of Product Development at AnyMind Group, developed it by himself in PHP, but soon he saw greater business potential and formed a team of engineers to work on this product – the development language was replaced by Python from there.
As shown in the following figure, PostgREST is responsible for reading and writing RDBs, and DB values are updated according to data updates from users or batch execution results, and subsequent AWS lambda functions are executed using these hooks.
As a development team, we organized around developing each Lambda function for cost optimization and easier division of works, but gradually bugs became increasingly noticeable. In my opinion, the reason was because of the absence of an architect.
– Uncontrollable and inconsistent data is created as more and more columns with similar names and ambiguous purposes are added to the table – Too many Lambda functions were added, and it is difficult to know which function to fix when we needed to change specifications – Processes that should normally be transactional are separated, resulting in little rollback processing
In the end, it became almost impossible to know when this data was updated by which Lambda function. (I found 240 Lambda functions in the past, and since there is no documentation for each of them, we’ll need to look at each process to find out what they do!)
At this point, we had no choice but to rebuild the product.
■ Product rebuild
It was right around this time that I joined the company, and I was initially responsible for another product – but due to the situation, I shifted my focus to this product. Also, a talented frontend engineer Koshimizu joined us at about the same time, so we started the rebuilding project with him as the frontend engineer, me as the backend engineer, and the aforementioned Takemoto as the product manager.
The influencer marketing business is still in its infancy and is changing rapidly with updates to each social media platform, so the product must be able to respond quickly to changes. Therefore, the goal of the rebuild was to create a high-quality system that can adapt quickly and an architecture that can scale with the team.
As a design policy, we decided to use DDD (Domain Driven Design) methodology, which I had experience with, to implement the system whilst considering how each entity should behave. Since product manager Takemoto was the tech lead of the initial system, he had an idea of all the functions of the system and how each of them should work, so we identified the use cases based on that and designed the domain model similar to the diagram below.
At this time, we initially tried to manage data from various social media platforms in a centralized manner, but there were differences between platforms (how to handle access tokens, metrics for various platforms), so we quickly switched to a policy of separating them.
As for the technology stack, I would have preferred JVM language since there was a lot of DDD knowledge, but since existing code was written in Python, I adapted to Python from the perspective of utilizing existing resources assets and minimizing learning costs for team members. At that time, Python already had a static type checking feature, and we decided that writing domain logic would be no problem as long as we tried it. As a result, the DB access area, which was not type-checked, occasionally produced bugs, but the domain logic was managed beautifully.
We use GraphQL to communicate with the frontend – where the frontend defines the data it wants and the backend prepares the data to match it. This allows the frontend to focus only on the UI without worrying too much about the backend implementation, and makes it easier for the backend to optimize by building Read Models as needed. Initially, we considered Relay, but since the platform client only needed a web frontend and did not need a generic schema, we took the form of preparing specialized endpoints for each page.
After making these improvements, it took us several months to release the new version, which included a completely revised DB design and infrastructure.
By the way, while we were focusing on the rebuild, things were going well on the business end, and of course, the existing system needed to still run existing campaigns.
We had a team of engineers and support staff from the initial build to maintain the legacy system whilst we were fixing bugs in the new system. One of the engineer Dat also shared with us where and how to bring in data from the legacy system when we migrated the platform. Without them, we would not have been able to migrate so smoothly – they were immensely helpful.
■ Setting up a system that scales
Now that we managed to rebuild a buggy system, the next step was to go on the offensive. We needed to increase the development speed in order to respond to requests for additional features that had been piling up for a long time, and that’s why we needed to increase the number of engineers and create a system that could scale.
On the frontend, the main focus was to release an eagerly awaited mobile version of AnyCreator, our mobile app for influencers. We chose Expo because we already used React for the web frontend, and since the domain logic was integrated into the backend, we were able to implement it using our collective web knowledge.
I remember that the only major difference from the web version was the use of native screen transitions for social media logins, other than the UI components.
The backend, on the other hand, was made into microservices because wanted to scale the engineering team working on this platform and utilize DDD more. We adopted Kotlin for the divided services and brought in members with Java experience, because it is easier to adopt DDD (in my observation, there are few teams that adopt static type checking and DDD in Python), and it is preferable to use a language that can easily handle asynchronous processing since we often handle external APIs.
In line with the shift to microservices, we introduced GraphQL Proxy, and in terms of infrastructure, we adopted GKE for ease of deployment and cost reduction of various subsystems, as well as for the deployment of infrastructure.
There are many more points that were implemented, but it is a long story, so I would like to introduce them in a future artice.
■ Two teams in place
With a larger team of engineers, and different subsystems having different technology stacks and domain knowledge requirements, we finally started to steer towards a two-team structure.
– Existing functionality centered on influencer marketing campaigns（Python and some Kotlin）
We split the backend team into two teams, so that they can develop independently in scrum meetings. On the other hand, the frontend team remains as a single team and join the meetings of both backend teams.
This is because both platforms – AnyTag (usually used by marketers) and AnyCreator (usually used by influencers) have a different division of functions between the frontend and backend.
By the way, during this time, my responsibility shifted towards managing the entire engineering team (across all AnyMind Group products), so I handed over the tech lead position to a new TechLead (Anton) and only showed up once a week to discuss the overall direction of the team.
■ Reflection and future plans
What I’ve shared above is an overview of the composition of the AnyTag and AnyCreator product development teams. In the past six months or so, we’ve had new tech leads who have become more familiar with their business domain, and they’ve been working on the system configuration, adding subsystems and messaging between systems.
Again, AnyTag and AnyCreator are not forever-stable systems, because it is dependent on external social networking platforms and will always respond to the updates of those platforms. Therefore, in order to be able to respond to changes in specifications at any time, we are aiming to create a code base and documentation that allows everyone us to scale even further.
When we first moved to a microservices architecture, we believed that it will be highly useful in both clarifying responsibilities and scaling the team, and fortunately, we have been able to do what we intended to do. We’re continuing to increase the number of members to speed up the development process even more.
—
AnyMind Group is developing various products including AnyTag and AnyCreator, and we are hiring more engineers and product managers. If you are interested in joining us, please select “product development” from the link below and see available positions.
https://anymindgroup.com/career/
We also have a blog written by engineers in AnyMind Group. If you’d like to read it, please click the links below:
- – English: https://medium.com/anymind-group/tech-blog/home
- – Japanese: https://qiita.com/organizations/anymindgroup
