Automating the Linux Quality Assurance with LISA on Azure
Introduction Building on the insights from our previous blog regarding how MSFT ensures the quality of Linux images, this article aims to elaborate on the open-source tools that are instrumental in securing exceptional performance, reliability, and overall excellence of virtual machines on Azure. While numerous testing tools are available for validating Linux kernels, guest OS images and user space packages across various cloud platforms, finding a comprehensive testing framework that addresses the entire platform stack remains a significant challenge. A robust framework is essential, one that seamlessly integrates with Azure's environment while providing the coverage for major testing tools, such as LTP and kselftest and covers critical areas like networking, storage and specialized workloads, including Confidential VMs, HPC, and GPU scenarios. This unified testing framework is invaluable for developers, Linux distribution providers, and customers who build custom kernels and images. This is where LISA (Linux Integration Services Automation) comes into play. LISA is an open-source tool specifically designed to automate and enhance the testing and validation processes for Linux kernels and guest OS images on Azure. In this blog, we will provide the history of LISA, its key advantages, the wide range of test cases it supports, and why it is an indispensable resource for the open-source community. Moreover, LISA is available under the MIT License, making it free to use, modify, and contribute. History of LISA LISA was initially developed as an internal tool by Microsoft to streamline the testing process of Linux images and kernel validations on Azure. Recognizing the value it could bring to the broader community, Microsoft open-sourced LISA, inviting developers and organizations worldwide to leverage and enhance its capabilities. This move aligned with Microsoft's growing commitment to open-source collaboration, fostering innovation and shared growth within the industry. LISA serves as a robust solution to validate and certify that Linux images meet the stringent requirements of modern cloud environments. By integrating LISA into the development and deployment pipeline, teams can: Enhance Quality Assurance: Catch and resolve issues early in the development cycle. Reduce Time to Market: Accelerate deployment by automating repetitive testing tasks. Build Trust with Users: Deliver stable and secure applications, bolstering user confidence. Collaborate and Innovate: Leverage community-driven improvements and share insights. Benefits of Using LISA Scalability: Designed to run large-scale test cases, from 1 test case to 10k test cases in one command. Multiple platform orchestration: LISA is created with modular design, to support run the same test cases on various platforms including Microsoft Azure, Windows HyperV, BareMetal, and other cloud-based platforms. Customization: Users can customize test cases, workflow, and other components to fit specific needs, allowing for targeted testing strategies. It’s like building kernels on-the-fly, sending results to custom database, etc. Community Collaboration: Being open source under the MIT License, LISA encourages community contributions, fostering continuous improvement and shared expertise. Extensive Test Coverage: It offers a rich suite of test cases covering various aspects of compatibility of Azure and Linux VMs, from kernel, storage, networking to middleware. How it works Infrastructure LISA is designed to be componentized and maximize compatibility with different distros. Test cases can focus only on test logic. Once test requirements (machines, CPU, memory, etc) are defined, just write the test logic without worrying about environment setup or stopping services on different distributions. Orchestration. LISA uses platform APIs to create, modify and delete VMs. For example, LISA uses Azure API to create VMs, run test cases, and delete VMs. During the test case running, LISA uses Azure API to collect serial log and can hot add/remove data disks. If other platforms implement the same serial log and data disk APIs, the test cases can run on the other platforms seamlessly. Ensure distro compatibility by abstracting over 100 commands in test cases, allowing focus on validation logic rather than distro compatibility. Pre-processing workflow assists in building the kernel on-the-fly, installing the kernel from package repositories, or modifying all test environments. Test matrix helps one run to test all. For example, one run can test different vm sizes on Azure, or different images, even different VM sizes and different images together. Anything is parameterizable, can be tested in a matrix. Customizable notifiers enable the saving of test results and files to any type of storage and database. Agentless and low dependency LISA operates test systems via SSH without requiring additional dependencies, ensuring compatibility with any system that supports SSH. Although some test cases require installing extra dependencies, LISA itself does not. This allows LISA to perform tests on systems with limited resources or even different operating systems. For instance, LISA can run on Linux, FreeBSD, Windows, and ESXi. Getting Started with LISA Ready to dive in? Visit the LISA project at aka.ms/lisa to access the documentation. Install: Follow the installation guide provided in the repository to set up LISA in your testing environment. Run: Follow the instructions to run LISA on local machine, Azure or existing systems. Extend: Follow the documents to extend LISA by test cases, data sources, tools, platform, workflow, etc. Join the Community: Engage with other users and contributors through forums and discussions to share experiences and best practices. Contribute: Modify existing test cases or create new ones to suit your needs. Share your contributions with the community to enhance LISA's capabilities. Conclusion LISA offers open-source collaborative testing solutions designed to operate across diverse environments and scenarios, effectively narrowing the gap between enterprise demands and community-led innovation. By leveraging LISA, customers can ensure their Linux deployments are reliable and optimized for performance. Its comprehensive testing capabilities, combined with the flexibility and support of an active community, make LISA an indispensable tool for anyone involved in Linux quality assurance and testing. Your feedback is invaluable, and we would greatly appreciate your insights.From Compliance to Auto-Remediation: Azure's Latest Linux Security Innovations
We are pleased to announce that the Azure security baseline through Azure Policy and Machine Configuration for Linux has moved to public preview, and we are expanding the capabilities with built-in auto-remediation feature (limited public preview). Customers face increasing pressure to comply with requirements set by governments, regulatory bodies, or specific industries. As their environments become more complex and hybrid, achieving and maintaining compliance on a large scale remains challenging and problematic. Failing to meet compliance goals can result in substantial business harm, including financial penalties and the potential loss of customers. Introducing enhanced audit and the new auto-remediation experience: Recognizing the above-mentioned challenges, Microsoft has developed a solution to help customers navigate these complexities at ease. The Azure security baseline for Linux offers compliance and built-in auto-remediation (limited public preview) features via Azure Policy’s Machine Configuration and Microsoft’s open-source Azure-OSconfig engine. The combination of these capabilities will ensure that security is embedded by design and compliance requirements are upheld, whether workloads operate in the cloud, on-premises, or in another CSP environment, through the Azure Arc platform. Thanks to the new approach we provide detailed information about the state of compliance and more accurate results with detailed descriptions with direct reference to the CIS rule definitions. Furthermore, the new architecture has enabled us to implement and provide automatic remediation capabilities against the security baseline providing a Linux-native experience for our customers when it comes to hardening. Microsoft has implemented a streamlined version of Linux security best practices, primarily based on the latest CIS (Center for Internet Security) Distribution Independent Linux benchmark. All the audit and remediation results are available and can be queried within the Azure Resource Graph Explorer for reporting and monitoring purposes. As security is Microsoft’s top priority, we will provide these capabilities at no additional cost to our customers, with charges only applying to the Azure Arc managed workloads hosted on-premises or other CSP environments. What’s next: At Microsoft we strive to continuously improve customer satisfaction - understanding that a one-size-fits-all approach is not feasible for hardening and security, we are committed to working with our customers throughout the preview process to improve the end-to-end experience. In addition to that, Microsoft is committed to evolve and further develop and deliver new security baseline contents to be fully aligned with the latest CIS standards across various Linux distributions and will collaborate with the relevant standard bodies to contribute to the standards, benefiting both the broader community and the wider industry. Stay tuned in this space for more information - exciting news to come in the upcoming months! What happens with the existing Azure security baseline for Linux capability: Every VM customer which has the “Linux machines should meet requirements for the Azure compute security baseline” policy definition assigned will be auto migrated by the Azure team in the upcoming months to the new policy definition. (audit only) We are going to do a gradual rollout of this enhanced capability. For the time being approximately 3-6 months post announcement, the existing policy will still be available and then it will be deprecated and removed from the Azure portal. Learn more: Sign-up form for the auto-remediation capability Read more about Azure Arc Check out the Azure osconfig’s GitHub repo Comparison between old and new baseline is attached to the blog List of supported operating systems (check the Linux distros in the table)Red Hat at Microsoft Ignite: Pioneering Innovation for the Cloud
Microsoft Ignite 2024 brought with it groundbreaking announcements, and Red Hat stood at the forefront, unveiling a series of innovations designed to empower businesses across industries. These announcements further strengthened the partnership between Red Hat and Microsoft, showcasing their joint commitment to delivering open-source solutions tailored for modern cloud workloads. In this blog, we’ll explore the key announcements made by Red Hat at Microsoft Ignite and how they align with the evolving needs of enterprises, from AI-driven workloads to high-performance computing, hybrid environments, and beyond. 1. Landing Zone for RHEL on Azure: Simplifying Migration One of the most exciting developments is the Landing Zone for Red Hat Enterprise Linux (RHEL) on Azure. This initiative provides organizations with a streamlined path to migrate their RHEL workloads to Azure. By leveraging the Landing Zone, businesses can: Simplify cloud adoption through pre-configured environments. Ensure compliance and best practices with built-in governance and security measures. Enhance operational efficiency by integrating with Azure-native tools. This offering caters to organizations at various stages of cloud adoption, empowering them to accelerate their journey to Azure with minimal friction. Learn more about this transformative capability here. 2. Red Hat JBoss EAP 8 on Azure: The Future of Java Workloads Red Hat also introduced Red Hat JBoss Enterprise Application Platform (EAP) 8 on Azure. This fully supported, jointly produced solution is a game-changer for Java developers aiming to modernize their applications in the cloud. Key benefits of this integration include: Seamless deployment of Java workloads in Azure environments. Enhanced support for microservices architecture. Access to Azure’s global scale, enabling developers to innovate faster and meet growing application demands. For developers and businesses relying on Java for critical workloads, this announcement solidifies Azure as a destination for innovation and modernization. Explore the details here. 3. HPC on Azure: Scaling Compute with RHEL The demand for high-performance computing (HPC) in industries like finance, healthcare, and engineering has never been greater. Addressing this, Red Hat has made significant strides in enabling RHEL for HPC on Azure. This development allows businesses to: Scale their compute capabilities dynamically. Leverage Azure’s robust infrastructure for intensive computational workloads. Integrate with RHEL’s ecosystem for consistent and secure performance. With this solution, Red Hat empowers organizations to meet the demands of data-heavy applications, ensuring they stay ahead in competitive markets. Dive deeper into RHEL for HPC on Azure here. 4. RHEL Meets Windows Subsystem for Linux (WSL): A New Era of Hybrid Environments In a landmark announcement, Red Hat Enterprise Linux is now available on Windows Subsystem for Linux (WSL). This collaboration bridges the gap between Linux and Windows environments, offering unprecedented flexibility to developers and IT professionals. Key highlights include: Access to RHEL’s trusted ecosystem on Windows devices. Streamlined development workflows for hybrid IT environments. Enhanced compatibility for organizations operating in multi-platform setups. This integration marks a significant step forward in breaking down barriers between operating systems, enabling developers to work seamlessly across their preferred environments. Learn more about this innovative solution here. 5. RHEL for SAP: Unlocking Value in Public Cloud Marketplaces SAP workloads demand stability, scalability, and high availability. Recognizing this, Red Hat announced RHEL for SAP in public cloud marketplaces, including Azure. With this offering, SAP customers can: Simplify procurement and deployment through Azure Marketplace. Leverage RHEL’s certified configurations for optimized performance. Reduce operational complexity with integrated support from Red Hat and Microsoft. This solution addresses the unique challenges of SAP workloads, empowering businesses to maximize their investment in SAP applications. More details can be found here. 6. RHEL AI: Empowering Generative AI Workloads As AI transforms industries, Red Hat unveiled RHEL AI, a solution designed to cater to generative AI workloads on Azure. This new offering provides enterprises with the flexibility and tools needed to harness the power of AI at scale. Key features include: Pre-configured RHEL environments optimized for AI/ML workloads. Integration with Azure AI services for accelerated deployment. A secure, scalable foundation for training and deploying AI models. This announcement underscores Red Hat’s commitment to staying at the forefront of innovation, empowering businesses to explore new frontiers in AI. Learn more about RHEL AI here. 7. Azure Red Hat OpenShift: Advancing Cloud Security with Confidential Containers Red Hat and Microsoft unveiled a significant advancement in cloud security with the public preview of Confidential Containers on Azure Red Hat OpenShift (ARO). This innovative solution brought hardware-based security measures to containerized workloads, offering unprecedented protection for sensitive data and applications. Key features included: Advanced memory encryption and secure workload execution using AMD SEV-SNP technology and Intel TDX capable instances Enhanced protection that safeguarded workloads even from cloud operator access Seamless integration with existing container deployment workflows and tools Zero additional costs during the preview period beyond standard Azure compute and ARO charges This solution was particularly valuable for organizations in healthcare, financial services, and regulated industries where data security is paramount. It also provided robust protection for sensitive AI/ML workloads. Organizations interested in enhancing their cloud security posture could explore this new capability through the preview program. To learn more, click here for more information. 8. Azure Red Hat OpenShift: Streamlining Enterprise AI Development Red Hat and Microsoft announced a significant advancement in their AI capabilities through Azure Red Hat OpenShift (ARO), addressing the challenges of deploying business-ready AI applications. This collaboration focused on integrating DevOps pipelines with data science workflows, enabling teams to prioritize AI model optimization over infrastructure management. Key features included: Pre-integrated DevOps and data science pipelines that streamlined deployment processes and accelerated time to value Enhanced AI performance capabilities through Red Hat OpenShift AI, Azure OpenAI, and RAG (retrieval-augmented generation) techniques GitOps deployment functionality utilizing ArgoCD templates for efficient production rollouts of AI models The integration demonstrated how organizations could leverage familiar tools and processes to accelerate their AI journey. To learn more, click here for more information. 9. Managed Identities Enhance Security in Azure Red Hat OpenShift Microsoft and Red Hat announced a significant security advancement for Azure Red Hat OpenShift (ARO) with the introduction of managed identity and workload identity support. This update marked a shift away from traditional long-lived credentials toward more secure, short-term privileged access mechanisms. Key features included: Implementation of eight distinct managed identities with built-in roles for different OpenShift components Short-lived credentials that eliminated the need for manual credential management Refined permission sets following the principle of least privilege Support for customer workload identities through Service Account Token Volume Projection and OIDC federation This enhancement addressed previous limitations where ARO required service principals with broad contributor-level access. The new approach provided granular control over permissions while improving security through time-bound access tokens. The announcement revealed plans for a preview release in early 2025, with multiple deployment options including an "all-in-one" command for streamlined implementation. To learn more, click here for more information. The announcements at Microsoft Ignite 2024 highlight the deepening collaboration between Red Hat and Microsoft. Together, they are shaping the future of enterprise IT by delivering innovative solutions that cater to the unique demands of modern workloads. To explore these innovations and how they can transform your IT landscape, visit Red Hat’s Ignite Page Stay tuned for more updates and insights as we continue to innovate together!
