Platform Engineering #9: How To Turn Kubernetes into an Internal Developer Platform

Learn how to turn Kubernetes into the best Internal Developer Platform - you'll learn this from our experience at Qovery.

Hey Folks 👋,

I'm Romaric Philogene, CEO and co-founder of Qovery (Internal Developer Platform), and this is my 9th post on my Substack. In my previous post, I explained how to design the ideal Internal Developer Platform.

Today, I’m excited to share with you how my team and I turned Kubernetes into an Internal Developer Platform that developers can easily leverage with the right Developer Experience. You can easily translate the concept of Qovery with your Internal Developer Platform. As explained in one of my last article, Kubernetes is now the standard to orchestrate containers. Bringing Kubernetes for Developers just make more and more sense, even if Kubernetes has not been designed for developers.

I encourage you reading why Kubernetes for Developers is the Next Big Thing.

—

To turn Kubernetes into a pleasant Internal Developer Platform platform for developers, Qovery acts as an intermediate layer that speaks the language of developers while converting their actions into Kubernetes operations. We’ve designed Qovery to provide a simple, versatile, powerful and and intuitive interface, allowing developers to manage their applications without needing to understand the intricacies of Kubernetes.

Qovery translates actions from Lara (developer) into native Kubernetes instructions - this is a simplistic view but you get the philosophy

Developing a product that abstracts Kubernetes enough to be usable by developers is an intricate process. It requires a high degree of knowledge and experience in infrastructure and engineering. The main challenge lies in building an intermediary layer that can bridge these two realms and allow them to work together efficiently while catering to their different expertise.

1. Integrating into the Developer environment

To create a genuinely developer-friendly product, it’s essential to have an in-depth understanding of how developers work. This involves knowing their preferred tools, workflows, pain points, and the programming languages and frameworks they use. By gaining insight into these aspects, we can develop a product that seamlessly integrates with the developer ecosystem and addresses their specific needs.

Qoverys integrates into GitHub, GitLab, BitBucket, any CI/CD platform, but is also usable from a web interface, CLI, API and even a Terraform Provider for the most advanced users - this is a simple view - Qovery integrates with more than that

One of Qovery’s key strengths is its ability to seamlessly integrate into the developer environment, ensuring a smooth and efficient workflow. Let’s use GitHub as an example to illustrate how Qovery simplifies the deployment pipeline for developers.

When working with GitHub, developers typically use branches to manage different versions of their codebase. Each branch corresponds to a specific feature, bug fix, or stage of development. Qovery integrates with this workflow by automatically deploying the changes pushed to a branch to the appropriate application in the right environment.

Example of How Qovery Works with GitHub

From a simple git push to a complete deployment on Kubernetes. The experience is transparent for the developer.

1. Automatic branch recognition: When a developer pushes their code to a GitHub repository, Qovery recognizes the branch and understands its purpose in the project. This allows Qovery to determine the application and environment to which the changes should be deployed.

2. Deployment pipeline management: Qovery takes care of the entire deployment pipeline, from building the application to deploying it in the correct environment. Developers don’t need to think about or manually configure the deployment process — Qovery handles everything behind the scenes.

3. Environment-specific deployments: Qovery supports deploying applications to different environments, such as development, staging, or production. Depending on the branch, Qovery deploys the changes to the appropriate environment, ensuring that the right version of the application is running in each context.

4. Automatic rollback: In case of a failed deployment, Qovery automatically rolls back to the previous stable version of the application. This feature ensures that any errors or issues introduced by new changes do not negatively impact the application’s stability or availability.

5. Monitoring and logs: Qovery provides developers access to monitoring tools and logs, allowing them to gain insights into their application’s performance and troubleshoot issues as they arise.

By integrating with popular developer tools like GitHub, Qovery streamlines the development and deployment process, making it more efficient and hassle-free. Developers can focus on writing code and implementing new features. At the same time, Qovery takes care of the deployment pipeline and ensures that their changes are correctly deployed to the right application in the appropriate environment.

Side note: of course - you can also integrate Qovery into your CI pipeline and leverage it to deploy and manage apps.

2. Infrastructure management

Building an intermediary layer between developers and Kubernetes requires a solid foundation in infrastructure management. This means thoroughly understanding Kubernetes and related technologies such as pods, ingress, services, nodes, cluster, <put your favorite CRD here 😅>… With this expertise, we can effectively simplify and automate the infrastructure management process for developers, enabling them to deploy and manage their applications without diving deep into the complexities of Kubernetes.

Every Developer-Friendly action are translated into Kubernetes actions

A key aspect of Qovery’s approach to simplifying Kubernetes for developers is its intuitive web interface. This interface allows developers to easily manage their applications, resources, and settings while Qovery translates their actions into appropriate Kubernetes instructions behind the scenes.

Here’s how Qovery’s web interface enables developers to interact with Kubernetes without having to learn its complexities:

1. User-friendly resource allocation: When a developer sets resource allocations, such as CPU, RAM, the number of instances, or storage, Qovery translates these choices into appropriate Kubernetes resource configurations. This process ensures that the application is deployed with the desired resources without requiring the developer to understand the intricacies of Kubernetes resource management.

2. Domain management: When a developer configures a custom domain for their application, Qovery translates this action into the necessary Kubernetes instructions, such as creating an Ingress resource or configuring a load balancer. This simplifies connecting a domain to an application, enabling developers to focus on their code rather than the underlying network details.

3. Educational tooltips and explanations: Qovery’s web interface includes tooltips and explanations for each input field and action, providing developers with context and understanding of the purpose and effect of their choices. By educating developers on the impact of their decisions, Qovery empowers them to make informed choices and gain a deeper understanding of the deployment process.

As developers interact with Qovery’s web interface, the platform automatically generates the necessary Kubernetes instructions to implement their choices. For instance, the (Rust) code used to generate the appropriate manifest on the Qovery Engine side looks like this:

fn on_create(&self, target: &DeploymentTarget) -> Result<(), Box<EngineError>> {
    let long_task = |_logger: &EnvProgressLogger| -> Result<(), Box<EngineError>> {
        //...
        let chart = ChartInfo {
            name: self.helm_release_name(),
            path: self.workspace_directory().to_string(),
            namespace: HelmChartNamespaces::Custom,
            custom_namespace: Some(target.environment.namespace().to_string()),
            timeout_in_seconds: self.startup_timeout().as_secs() as i64,
            k8s_selector: Some(self.selector()),
            ..Default::default()
        };

        let helm = HelmDeployment::new(
            event_details.clone(),
            self.to_tera_context(target)?,
            PathBuf::from(self.helm_chart_dir()),
            None,
            chart,
        );

        helm.on_create(target)?;

        delete_pending_service(
            target.kubernetes.get_kubeconfig_file_path()?.as_str(),
            target.environment.namespace(),
            self.selector().as_str(),
            target.kubernetes.cloud_provider().credentials_environment_variables(),
            event_details,
        )?;

        Ok(())
    };

    execute_long_deployment(ApplicationDeploymentReporter::new(self, target, Action::Create), long_task)
}

This automation removes the need for developers to write complex Kubernetes configuration files or learn the Kubernetes command-line interface.

3. Balancing abstraction and control

One of the most critical aspects of building an intermediary layer between developers and Kubernetes is striking the right balance between abstraction and control. We must provide enough abstraction to make Kubernetes accessible to developers without requiring them to become infrastructure experts. At the same time, we need to ensure that developers retain sufficient control over their applications and can fine-tune their deployments as needed.

An intuitive way to give back control over simplification

To overcome this challenge, Qovery offers “Advanced Settings” where developers can modify eventually every parameter for underlying resources. For example, if a developer needs to change the `max_body_size` parameter for the ingress controller used by their application, they can do so through these settings. This feature gives developers control over the more advanced aspects of their applications while still keeping the overall experience simple and user-friendly.

Advanced Settings are a gentle introduction for developers who want to learn more about the underlying infrastructure components. By making changes to specific parameters, developers can gain insights into how different components work together and how their applications are affected by these changes. This gradual education process allows developers to become more familiar with the inner workings of their applications without being overwhelmed by the complexity of Kubernetes.

Nginx 502 error because of a too large header buffer size — Advanced Settings to the Rescue 🦸‍♀️

An excellent example of this can be seen in a our community discussion thread, where a developer had to change the `proxy_buffer_size_kb` parameter for the NGINX ingress controller to a higher value. The developer encountered a 502 Bad Gateway error with their Next.js application using Auth0 for authentication. By adjusting the parameter, they resolved the issue, demonstrating the value of Qovery’s Advanced Settings and how it empowers developers to troubleshoot and optimize their applications.

4. Continuous improvement and adaptation

Developers’ needs are constantly moving. To ensure that our Internal Developer Platform remains a valuable tool for developers, we must continuously adapt and improve the product to meet these changing needs. By staying up-to-date with the latest infrastructure management and software development trends, we can keep Qovery at the forefront of the industry and maintain its position as a powerful, user-friendly solution.

Linkedin post from one of our customers (CTO @ Hyperline) about our support and continuous improvements

Here are some ways in which we maintain a strong connection with our users and fosters continuous improvement:

1. Direct engineering support: we provide direct engineering support to our users, ensuring they receive timely assistance and guidance from our engineers who build the product. This hands-on support not only helps users resolve issues quickly but also enables the Qovery team to gather valuable feedback and insights that can be used to enhance the product.

2. Public roadmap: we maintain a public roadmap that showcases planned improvements, fixes, and features. This roadmap is not just for show — user feedback and needs are genuinely considered when prioritizing updates. By involving our community in the decision-making process, Qovery can better align its development efforts with the needs of its users.

3. Regular changelogs: At the end of each two-week sprint, we provide a changelog detailing the product’s updates. These changelogs inform users about the latest improvements and ensure they are always up-to-date with the platform’s capabilities and features.

4. Educational content: Writing articles like this is another way we stays close to our users. By sharing insights, experiences, and best practices, sd educate our users and fosters a sense of community and trust. This open line of communication ensures that users feel heard and valued, which is essential for building a product that meets their needs.

By staying closely connected to our users and continually evolving based on their feedback, we can build a product that caters to the needs of both engineering and platform engineering teams. This commitment to continuous improvement and adaptation ensures that Qovery remains an invaluable tool for developers as technology and user requirements change over time.

How applicable is to you?

LuiNow you have a better understanding on how we have built Qovery to transform Kubernetes into an Internal Developer Platform. Even if not completely applicable to the way that you will build an Internal Developer Platform - you learnt that the process and the way we iterate with our users is one of the most important aspect of building a used and loved Internal Developer Platform.

I hope you enjoy this post. If you have any questions - please feel free to drop your comments :) I’ll be happy to respond.

Let's revolutionize Platform Engineering by putting developers first. Subscribe now to join me on this exciting journey!

---