Nowadays, you can hear about generative AI and the latest large linguistic models from every corner. The speed at which Gen AI is being modified and invading every part of our lives is shocking. That's why many business leaders who want to be competitive and on top of cutting-edge technologies pivot to Gen AI and find ways to quickly respond to the pressures of these changes.

These changes also affected our ML Platform, a self-hosted Kubernetes Platform that enables 25 data science teams to work on it. Our task was to fine-tune the ML Platform in the shortest possible time—about three months—to enable all Data Science teams to quickly research, test, and develop their Gen AI and LLM ideas rapidly and smoothly.

We’ve encountered unexpected hurdles throughout our journey — from grappling with incompatible libraries to navigating through incoherent LLMs, daunting cost risks, and more. In my talk, I will share our main learnings, challenges, and pitfalls, as well as where it all led and how, against all odds, we could quickly build a product on our platform, allowing our data teams rapid testing of GenAI use cases.