Hey, Andrey is back, and this is the second part of the "Accountability App" story. In the first part, I discussed the idea and the tech design that we needed to implement.
In the second part, I will elaborate on the execution and reflection phases. I will discuss the release train we built, the challenges we faced with our architecture, and what we learned as a team. As the tech lead, I gained valuable experience from this project that is still helping me out every day in engineering.
How the first months went by
It took us approximately 2-3 months to go from the product idea to launching on the market. We acquired our first users 3 months after starting development. However, the app was not entirely ready for these initial users. To be completely honest, it was only after six months that our app became robust enough to emerge from beta.
We had been working in one-month iterations, just as our CEO had been funding the development, with the expectation that each iteration would bring a real product increment.
Eventually, we established two-week sprints, and in almost every sprint we were able to release something to the market.
The overall app life timeline for the first half-a-year
However, churn due to the lack of robustness was a significant challenge during the first few months after the MVP release. The accumulated harm caused by this issue ultimately led to the product's downfall.
As expected, most coaches were not willing to be beta testers. Therefore, we had to establish almost seamless communication between our early adopters and the CTO/Dev team. In fact, a few times we even had been delivering notifications manually when our algorithm failed to perform its job correctly!
How We Spun Out Our Release Train
Market release timeline
The first challenge we faced setting up our post-MVP processes was the difference between Mobile development and web one. When you build stuff for mobiles you should be ready for the long approvement review and potential push-backs, especially from the Apple team. We needed to plan things ahead, and never rely on a fast approvement in case of urgent hotfixes, so each feature could be easily stuck for a week - we designed our release train correspondingly.
Workaround for these “delayed releases” became Expo direct deployment (by swapping .js files right in the working app), it was an amazing way to mitigate risks, and we saved a few ongoing programs by hot-fixing scheduling issues right on time instead of going through a long and clumsy process of updating via the store.
In addition, we were making intermediate releases - mostly change request ones, copy tweaks, and so on.
Engineering the train
As a foundation, we chose the good old 2-week Scrum as we had before. The 1-month iteration pace requested by our founder was perfect to fit 2 sprints into, and eventually, we were spending about 1.5 (3 weeks) for actual feature development, and the rest 1 week for tech debts and regression run before the next release
Speaking of 1-week regression - yes, we had failed to automate our tests, since React Native didn't have (or we just didn't find) a good option of doing it seamlessly, so we established the comprehensive testing before the release and were trying to optimize it as much as possible. It was a straightforward workaround to prevent issue leaks, however, never worked fully - leaks still were happening.
Also, we’d been doing a regular backlog refinement to find and reduce ongoing defects, Our goal was a zero-known-defect state, but we never actually reached that.
Along with refinement we usually prepare the next scope from the CEO’s ideas, customer complaints, and our own technical points. That’s how we were planning our work, and it let us be on time and ship the software no matter the lack of expertise and constant pivots.
Finally, apart from the main pipeline we were working closely with our customers to reduce the delay between problem spotting and its fix. I was often debugging issues right on the users' accounts, and the ability to do that tailored to architecture was a good choice.
That’s how our release train looked like, sometimes we broke our terms but in 80% we followed the system:
And what we had as a result
As a tech lead and, eventually, Project manager I was responsible for the timeline and engineering excellence around the project, I had a lot of work, and a lot of things to learn. I felt like I got out of the 1-year venture as a new, mature engineer and tech lead.
We brought a huge pile of experience out of the whole enterprise, what we have learned from the whole enterprise is what worked and what didn’t:
What worked well
Eventually, we were able to establish a fair delivery pipeline and robustness and could implement features within a single iteration. I have since taken many of the solutions developed for this app and applied them to my daily work.
I learned many technical skills such as Mobile APIs, MQTT (for chats), and offline cases during this project. This experience has allowed me to become a generalist developer, capable of working on any project, language, or platform, rather than specializing in one specific area of expertise.
Upon returning to cozy SaaS web development, we found that our processes had become much more reliable and mature. Mobile development proved to be a challenging task.
Finally, I learned not only how to deal with mistakes, but also how to defend your decisions (even bad ones) in front of customers. I had a few tough conversations with the founder after some things went wrong.
What Didn't
The biggest technical challenge we faced was scheduling notifications. The Notifications API wasn't intuitive, and we had to pivot three times before our notifications delivery became robust enough to avoid cringe-worthy errors every other release.
One lesson that has been emphasized in many places is that technical design should be as simple as possible. We should leave as many decisions undone as we can afford while still shipping features. This is especially important in mobile development, where different users may have different versions and we can't afford to break backward compatibility.
Never release a raw beta version to users who are not willing to be beta testers. The churn we faced was actually detrimental to the product's success. Although it was not our decision, it's a lesson I took away and will remember for the rest of my development career!
Conclusion
This project helped me grow from a developer to a tech lead/engineering manager. Through this experience, I learned that development is only successful when it is managed properly and when clear priorities are established in the backlog.
Following this huge project, we launched a few smaller MVPs which went much smoother. Stay tuned for more upcoming posts on the topic!
Please share your own stories of building a startup. Every story is unique, which is what makes such ventures especially interesting!
