After acquiring a handful of AWS certifications and migrating an early prototype of Perfect Form, my computer vision-powered lifting form analysis platform, from a local microservice to the cloud, I heard about an upcoming opportunity to share what I had learned. AWS Community Day was scheduled to take place that summer in my backyard and was looking for speakers. I learned a lot from all of the hours I had poured into my project. And more importantly, I learned a lot about what I would want to steer other developers looking to follow my path away from. In particular, I was interested in helping builders looking to follow a similar evolution starting from a trained machine learning model and adding in the necessary software engineering and then solutions architecture to cost-effectively develop solutions leveraging the full benefits of AWS.

I was passionate about sharing my experience and excited to use a PTO day as well as many evenings preparing for the talk because I felt let down by the existing resources. In my experience, there were good resources for cloud engineering absent machine learning models and good resources for deploying machine learning models absent cloud engineering. I was excited to attempt to fill in this perceived gap. While neither of these areas were particularly new, the best practices for combining the two still seemed very much in motion. The audience, at least as I envisioned it, seemed like it could be a good mix of folks with a cloud background who might be curious about incorporating machine learning into their work as well as people with more traditional machine learning backgrounds looking to use the cloud. Although I would not consider myself an expert, I believed that I acquired some pertinent lessons to share with both of these groups, so I excitedly committed to submitting a talk proposal.

After some reflection, I chose the title “Avoiding Common Pitfalls with Hosting Machine Learning Models” for my talk. Just like in most of solutions architecture, I do not believe that there is a single general solution that can be prescribed absent the context of the particular system. But I do believe that there are some trajectories and temptations that are often incorrect. While I may not know exactly the best way to architect every machine learning solution in the cloud, I felt confident in advising against certain directions based on learning these lessons firsthand. So I reviewed my experience, making notes of the sharp edges that I encountered, and organized my thoughts into a proposal, which ultimately was accepted.

As I started creating slides for my talk, I quickly realized how fast thirty minutes would fly. Consequently, I realized that I had to focus on a specific slice of the combination of machine learning and cloud development if I were to talk at any depth. I was most interested in addressing the knowledge gap around machine learning systems of intermediate complexity. For simple systems, there are tons of tutorials and examples. Additionally, there are lots of resources around SageMaker and all of its functionality designed to make deployments and monitoring very easy by relying on lots of abstraction. On the other end of the difficulty curve, every production system I’ve worked with professionally is relying on custom container declarations orchestrated in Kubernetes. At this level of complexity, there are fewer resources available, but far from zero: there are lots of system diagrams for all kinds of machine learning solutions and discussions around best practices. The real problem is the lack of resources for the very large region of complexity between these two ends of the spectrum. And worse, there are even fewer good resources helping developers take a simple system and build in more complexity. Where are all of the steps between single-line deployments in SageMaker and managing Kubernetes clusters for serving models?

A lack of intermediate resources makes learning harder than necessary.

I crafted my talk to try to answer this question as thoroughly as possible within my allotted time slot. I will not attempt to reproduce my lengthy talk in this article. Unfortunately, there were technical issues with recording the videos of presentations, so my talk was among the many that were not successfully recorded. As a small consolation, I decided to make the slides publicly available, so feel free to look through them if you are interested.

Although I believe that my talk turned out very well, I do not think that it fit the audience of this conference. Outside of a few curious students, almost every attendee with whom I spoke was working directly with AWS exclusively, which perhaps should have been obvious to me. When I say “exclusively,” I do not mean in the sense that they champion AWS over a multi-cloud approach. Rather, I mean that most of the attendees only wrote code to run in the cloud. My entire target demographic of someone looking to push a project as far as possible locally to maximize developer velocity and minimize cost before migrating to the cloud was not well represented at thi conference. Most of the people attended the conference because their work paid for them to go learn about a specific AWS service that they should be using or spread the word about the exciting things happening at their company. This seems obvious in hindsight, but most people at the conference were not spending way too many late nights and weekends trying to build solutions on AWS looking to use their PTO to connect with like-minded individuals.

To be clear, I blame my read of the target demographic of the conference, not AWS Community Day. The conference had many interesting sessions and speakers that were entirely appropriate for an AWS-sponsored conference. And despite my impedance mismatch, every person with whom I interacted was very welcoming, and several people in the audience were kind enough to ask me a few questions and even find me after my talk for further discussion even though my talk probably missed the mark. I did make a few connections, too, which is always fun as someone who works remotely in a city without a large tech community. I am still proud of the talk that I created and delivered, and I do not regret spending the day getting to learn more about AWS. Perhaps someone stumbling on this article will find the contents of my talk a useful guide for their future project.