IBM has made a significant stride in the realm of artificial intelligence by announcing that its Granite models—a combination of open-source and proprietary models tailored for business applications—are now accessible through Docker Hub. This move also includes the availability of Red Hat’s InstructLab model alignment tools. Docker Hub is a popular platform where developers can share and find containerized applications, making it an ideal place for such models that are designed to streamline development processes.
These Granite models are crafted to cater specifically to developers, allowing them to seamlessly access, deploy, and scale their AI applications. This development will be formally introduced during the AI segment of the keynote at the IBM TechXchange conference on October 22. Attendees will have the privilege of witnessing firsthand how these models can enhance AI-driven application development across various programming languages.
The Significance of Granite on Docker Hub
IBM’s decision to offer Granite models on Docker Hub is backed by their commitment to data transparency, model alignment, and security. These models represent a major advancement in the field of natural language processing (NLP). Being available under the Apache 2.0 license, they empower developer teams to incorporate generative AI into essential applications and workflows seamlessly.
Granite models are known for their exceptional performance in coding and specialized language tasks, operating with lower latency and at a fraction of the computational resource requirement. This efficiency translates to cost-effective inference processes, enabling developers to experiment, build, and scale AI applications both on-site and in the cloud while remaining within budgetary constraints.
Here’s what developers can expect:
- Simplified Deployment: By simply pulling the Granite image from Docker Hub, developers can get started in minutes.
- Scalability: Docker provides a lightweight, efficient method for scaling AI and machine learning applications. Developers can run multiple containers on one machine or distribute them across several machines in a cluster, facilitating horizontal scalability.
- Flexibility: Customize and extend the model as needed without the hassle of underlying infrastructure concerns.
- Portability: Creating Docker images once and deploying them anywhere helps eliminate compatibility issues and minimizes configuration needs.
- Community Support: Developers can leverage the extensive Docker and IBM communities for support, extensions, and collaboration opportunities.
In addition to Granite models, Red Hat has also made its InstructLab model alignment tools available on Docker Hub. These tools enable developers to adapt pre-trained large language models (LLMs) with significantly less real-world data and computing resources compared to other techniques. InstructLab is model-agnostic, meaning it can fine-tune any LLM by providing additional skills and knowledge.
Getting Started with Granite
Developers can access a variety of images on Docker Hub to get started with Granite:
- InstructLab: Geared towards desktop or Mac users, this image is perfect for those looking to explore InstructLab without needing specialized hardware. It’s ideal for prototyping and testing before scaling up.
- InstructLab with CUDA Support: Designed for full training workflows on GPU-equipped Linux servers, this image accelerates synthetic data generation and training using NVIDIA GPUs.
- Granite-7b-lab: Optimized for model serving and inference on desktop or Mac environments using the Granite-7B model, this image allows efficient and scalable inference tasks without needing a GPU, suitable for smaller-scale deployments or local testing.
- Granite-7b-lab with CUDA Support: For GPU-equipped Linux servers, this image supports faster model inference and serving via CUDA acceleration, ideal for high-performance AI applications where quick response times are crucial.
Running IBM Granite Images from Docker Hub
IBM Granite provides a comprehensive toolset for building and managing cloud-native applications. To pull and run an IBM Granite image using Docker and the command-line interface (CLI), follow these steps:
Authenticate to Docker Hub
Enter your Docker username and password when prompted to authenticate.
Pull the IBM Granite Image
Pull the IBM Granite image from Docker Hub. There are two versions available:
redhat/granite-7b-lab-gguf: For Mac/desktop users without GPU support.redhat/granite-7b-lab-gguf-cuda: For Linux systems with NVIDIA CUDA support.Running the Image in a Container
Start a container with the IBM Granite image, which can be initiated in two modes: CLI (default) and server.
To launch the container in CLI mode, use the following command:
bash<br /> docker run --ipc=host -it redhat/granite-7b-lab-gguf<br />
This command opens an interactive bash session within the container, enabling you to use the tools available.To run the container in server mode, execute this command:
bash<br /> docker run --ipc=host -it redhat/granite-7b-lab-gguf -s<br />
For detailed guidance on using IBM Granite Models, refer to the IBM Granite documentation.Join Us at IBM TechXchange
The official announcement of Granite on Docker Hub will take place at the IBM TechXchange Conference, scheduled from October 21-24 in Las Vegas. Eli Aleyner, the head of technical alliances, will conduct a live demonstration during the AI track of the keynote. Oleg Šelajev, Docker’s staff developer evangelist, will illustrate how application developers can test their GenAI applications using local models. Furthermore, attendees will learn how Docker’s collaboration with Red Hat is enhancing developer productivity.
The availability of Granite on Docker Hub marks a pivotal milestone in democratizing access to advanced AI models. It is an exciting development, and we look forward to seeing how development teams will utilize Granite’s capabilities to drive innovation and tackle complex challenges.
Stay tuned for more updates, and as always, happy coding!
For more information, visit IBM’s Granite page.
For more Information, Refer to this article.
