Cost-Aware, Scale-to-Zero AI Platform on Kubernetes
A multi-tenant AI platform using KEDA, shared LLM gateways, and per-service autoscaling to minimize idle infrastructure cost.
KubernetesKEDALiteLLMSSO/IAMScale-to-ZeroMulti-tenantCost Optimization
Available for impactful opportunities
Bayu Dian Nugroho
AI Platform Architect
I design production AI systems that scale when needed—and cost nothing when they don't.
Built enterprise AI platforms with Microsoft Entra integration, serving 100+ weekly userswith strict security and compliance.
10+
Years Experience
Indonesia
Location
3+
Cloud Platforms
Platform EngineeringCost OptimizationProduction AI
Systems That Other Engineers Build On
Trusted with cost, reliability, and failure domains. Platforms—not just features.
10+
Years
E2E
Ownership
About
I design systems that survive production constraints
Senior Backend Engineer and AI Platform Architect with 10+ years of experience designing cost-aware, production-grade AI systems. Specialized in building multi-tenant platforms that scale to zero when idle—minimizing infrastructure costs while maintaining enterprise-grade reliability. Led architecture and implementation of AI platforms serving enterprise clients with strict security and compliance requirements. Strong focus on Kubernetes-native architectures, event-driven systems, and observability-driven operations.
Bayu Dian Nugroho
Senior Backend Engineer | AI Platform Architect
Indonesia · Remote friendly
Languages
Indonesian (native)English (professional working proficiency)
Interests
Platform EngineeringCost OptimizationAI SystemsKubernetesMulti-tenant Architecture
Cost Ownership
Designed scale-to-zero AI platforms that eliminate idle compute costs.
Platform Thinking
Built multi-tenant systems with per-service cost attribution and shared LLM gateways.
Production Reality
Operated systems under real constraints: unpredictable traffic, strict auth, budget limits.
End-to-End Ownership
From design reviews and tradeoff decisions to production incidents and evolution.
My Approach
My approach is opinionated but pragmatic: favor simple scaling models over clever ones, isolate failure domains early, and treat cost as a first-class architectural constraint—not a finance problem discovered later. I'm most effective when owning systems end-to-end: from architecture design and technology decisions to production incidents and long-term evolution.
I've led the design of cost-aware AI platforms that scale individual services to zero when idle, route traffic through shared LLM gateways for usage control, and enforce authorization across shared data sources using SSO and IAM.
Skills
Skills & technologies
A cross-functional toolset spanning backend systems, DevOps, AI/ML, and modern web development.
Backend Engineering
- Go
- Python
- PHP
- Node.js
- C#/.NET
- C/C++
- Rust
Cloud & DevOps
- Azure
- AWS
- GCP
- Tencent Cloud
- Kubernetes
- Docker
- Podman
- KEDA
- ArgoCD
- Jenkins
- GitHub / GitLab CI
- Terraform
Frontend
- JavaScript
- React
- Vue
- Flutter
- HTML5
- CSS3
- Tailwind CSS
- Bootstrap
Data & Messaging
- PostgreSQL
- MySQL
- CockroachDB
- MongoDB
- OracleDB
- Vector DB
- MinIO
- Redis
- Kafka
- NSQ
- RabbitMQ
AI / ML Systems
- AI Agents
- LLM Integration (OpenAI, Azure, Google, Anthropic)
- Prompt Engineering
- RAG
- LangChain
- Azure AI Search
- LiteLLM Gateway
- YOLOv8
- OpenCV
- Document processing (PDF/Office)
- Search & ranking (TF/IDF, fuzzy)
- ElevenLabs Voice Synthesis
Real-Time & Integration
- WebSocket
- Rive Animation
- Twilio/WhatsApp API
- Moodle LMS Plugin Development
- Microsoft Entra (Multi-tenant SSO)
- Azure Transcribe + Diarization
Observability & Tools
- Grafana
- Prometheus / Mimir
- Loki
- Tempo
- OpenTelemetry
- Git
- Linux
Experience
Experience timeline
A decade of building scalable systems, microservices, and production AI/ML workloads.
PT Berlian Sistem Informasi
Lead Architect & Senior Software Engineer
May 2024 – Present
- EIRA - Enterprise AI Platform: Architected multi-tenant platform with Microsoft Entra auth, 6 AI features (Document QnA, Translation, MoM Extraction), deployed on AKS with per-service per-user access control. Reduced meeting documentation by 70%.
- DESTA - WhatsApp AI Assistant: Built RAG-powered dealership chatbot with NLP routing via Twilio. 95%+ intent accuracy, 60% faster response time, thousands of monthly conversations.
- AI Trainer - Role-Play Platform: Launched real-time training platform with WebSocket, Rive avatars, ElevenLabs voice. 100+ weekly active users, Moodle LMS integration.
- PTC - Customer Intelligence: AI-powered insight extraction with Azure Transcribe + Diarization. 50% reduction in Service Advisor prep time, 500+ conversations monthly.
- Scale-to-Zero AI Platform: Designed cost-aware KEDA-based platform with LiteLLM gateway. 10x cost reduction for document retrieval, 85% faster processing.
- Built Go-based observability gateway (NSQ + Fasthttp) streaming to Loki, Tempo, Mimir for AI pipeline monitoring.
GoPythonReactFlutterPostgreSQLAzure OpenAIAzure AI SearchMicrosoft EntraAKSKEDALiteLLMWebSocketRiveElevenLabsMoodleTwilioGeminiSCCIC ITB
Software Engineer
Sep 2023 – Apr 2024
- Integrated YOLO-based face and object detection into backend services for production video analytics workloads.
- Deployed services across cloud and on-prem environments using Docker, Airflow, and Nginx with automated workflows.
- Built backend services in Go and contributed across Node.js, Python, and PHP stacks in small engineering teams.
GoPythonNode.jsPHPYOLODockerAirflowNginxPT Neuronworks Indonesia
DevOps Engineer
Jan 2023 – Sep 2023
- Implemented Kubernetes-based orchestration to reduce infrastructure costs and improve deployment reliability.
- Containerized applications with Docker and set up GitLab CI/CD pipelines for automated deployment.
- Configured monitoring and alerting with Prometheus, Grafana, and related tooling.
KubernetesDockerGitLab CIPrometheusGrafanaLinuxPT Neuronworks Indonesia
Junior System Analyst
Jan 2020 – Jan 2023
- Analyzed and documented business and system requirements across multiple projects.
- Led migration of monolithic applications to microservices using Docker, Kubernetes, and MinIO.
- Implemented geolocation-based fraud prevention features for financial applications.
DockerKubernetesMinIOPostgreSQLMySQLPT Neuronworks Indonesia
Software Developer
Mar 2013 – Dec 2019
- Developed web applications using PHP and Zend Framework for enterprise clients.
- Implemented ETL-based data analytics pipelines for business intelligence.
- Contributed to documentation and code standardization efforts.
PHPZend FrameworkMySQLETLLinux
Projects
Systems I designed
Production platforms with real constraints, tradeoffs, and outcomes.
Featured Systems (7)
Problem
Companies need AI assistance for daily operations but face challenges with data privacy, compliance, managing access across departments, and providing consistent AI capabilities while maintaining strict security boundaries.
Constraints
- Multi-tenant platform with per-service cost visibility
- Strict authorization across shared data sources via Microsoft Entra
- Granular per-service per-user access control
- Azure Kubernetes Service deployment with minimal standby resources
Key Design Decisions
Multi-tenant via Microsoft Entra
Single identity provider serving multiple business units with isolated data access and per-service per-user authorization.
Modular Microservices Architecture
Each AI feature deployed independently on AKS with independent scaling, reducing blast radius and enabling granular access control.
Shared Azure OpenAI with Cost Attribution
Centralized LLM access with rate limiting and per-tenant cost tracking, preventing usage sprawl while optimizing resource utilization.
Tradeoffs Accepted
- Complex identity management across multiple business units
- Increased operational surface area with granular access policies
Outcomes
- Enabled enterprise-wide AI adoption with 100% data privacy compliance
- Reduced manual meeting documentation by 70% through MoM extraction
- Established reusable AI service patterns adopted across organization
Other Projects (6)
Distributed Document Processing Platform
End-to-end platform for bulk document ingestion, extraction, and semantic search. Reduced processing time from ~2-3 minutes to 10-30 seconds (~85% improvement).
GoPythonRedisMicroservicesRAGTF/IDFKubernetes
Real-time Image Processing Pipeline
WebSocket-based real-time monitoring stack for image-processing workloads, integrated with Flutter clients and observability tooling.
GoWebSocketNSQLokiTempoMimirKubernetes
Microservices Migration for Legacy Systems
Migrated monolithic PHP applications to containerized microservices with Kubernetes and MinIO-backed storage.
PHPDockerKubernetesMinIOMicroservices
Video Analytics with YOLO
Integrated YOLO-based video analytics for face and object detection, deployed across cloud and private infrastructure.
YOLOPythonDockerAI/ML
OTel + Grafana Observability Stack
Production-ready OpenTelemetry demo with Go service, OTel Collector, Grafana dashboards, Tempo distributed tracing, and Prometheus metrics. Full observability pipeline from instrumentation to visualization.
OpenTelemetryGrafanaPrometheusTempoGoObservabilityDocker ComposeOTLP
Education
Education
Formal education that built strong foundations in information systems and software engineering.
ST INTEN
Bachelor Degree, Information Systems
2011 – 2015
- Built early web applications and internal tools as part of coursework and side projects.
- Developed strong foundations in information systems, databases, and software engineering.
Contact
Let's collaborate
Interested in backend, DevOps, or AI/ML projects? Reach out and let's explore how we can work together.