Choosing the Right Terraform Workflow: VCS, CLI, or API?

NewsChoosing the Right Terraform Workflow: VCS, CLI, or API?

In today’s fast-paced digital landscape, the primary objective for many organizations is to streamline their software development workflows and tools. This not only simplifies operations but also enhances security and stability. When it comes to provisioning workflows, businesses aim to establish "golden workflows and components." However, flexibility is crucial for applications that require specific customizations, thus allowing seasoned developers and operators some freedom to provision infrastructure beyond the standard guardrails.

HCP Terraform, previously known as Terraform Cloud, offers this much-needed flexibility through API-driven workflows. These workflows provide customization options but are inherently more complex than UI/Version Control System (VCS) or Command Line Interface (CLI)-driven workflows. It is advisable for experienced Terraform users to manage these workflows. This article will explore the three types of HCP Terraform run workflows—CLI, UI/VCS, and API—delving into their use cases and benefits.

Choosing the Right HCP Terraform Run Workflow

The journey of infrastructure provisioning varies across organizations, influenced by business needs and the skills of administrators and developers. No single workflow fits all scenarios. Teams must assess and adopt processes that best align with their specific objectives and dynamics. Here are key factors to consider when tailoring workflows for your team:

  • Setup efforts
  • Maintenance efforts
  • Security concerns
  • Architectural and process complexity

    The Three HCP Terraform Workflows

    HCP Terraform presents three workflows for managing Terraform runs, with each determining the timing and method of Terraform executions.

    UI/VCS

    The UI/VCS-driven run workflow is the simplest and most user-friendly method for HCP Terraform runs. This setup allows HCP Terraform to connect with major VCS providers such as GitHub or Bitbucket. Upon connection, HCP Terraform automatically registers webhooks with your VCS provider when you create a workspace. Any new commits merged into the branch of a workspace’s linked repository automatically trigger a Terraform run. Additionally, a speculative plan is performed when a pull request is opened against that branch.

    Use This When

  • You require simple traceability and transparency.
  • You wish to implement a GitOps workflow for managing large-scale deployments.
  • You seek an easy-to-set-up, collaborative provisioning workflow.
  • You aim to provide users with a straightforward self-service workflow.
    Setup Tasks

  • Access: Ensure users have access to the VCS and HCP Terraform with the necessary permissions.
  • Define the repository architecture: Set up repositories through your VCS for internal teams to store Terraform configuration files.
  • Integration: Link your Terraform organization to your chosen VCS, such as using GitHub Apps or GitHub OAuth.
  • Create an HCP Terraform workspace: Follow the UI guidance to create a workspace linked to one branch of a VCS repository, ignoring changes to other branches.
    Maintenance Tasks

  • Repository branch management: Maintain disciplined branching and versioning, regular merging, conflict resolution, and alignment with infrastructure changes.
  • VCS provider integration management: Handle user access to different repository sets and plan for VCS provider maintenance if personnel changes occur.
    Security Concerns

  • Users with repository access can trigger Terraform runs.
    Architectural and Process Complexity

  • This workflow is highly automated, promoting collaboration and thorough review processes.
  • It scales easily for teams and multiple environments.

    CLI

    The CLI-driven run workflow utilizes Terraform’s CLI tools for remote execution in HCP Terraform. This method brings Terraform’s collaboration features into play, drawing strengths from both VCS and API-driven workflows. It is ideal for developers accustomed to using the Terraform CLI and can integrate with existing CI/CD pipelines.

    However, the learning curve is steeper if you haven’t used the various CLI tools like terraform or tfci. Familiarize yourself with these tools before adopting this workflow. A CLI-driven run can be initiated by simply running standard terraform plan and terraform apply commands, with progress viewable from your terminal or the HCP Terraform UI.

    Use This When

  • You are comfortable with the Terraform CLI.
  • You want to enable additional provisioning commands.
  • Small team members want to test configurations locally.
  • You want to start with CLI-driven workflow and add VCS connections later.
  • Your workspaces are not linked to any VCS repository.
    Setup Tasks

  • Build: Install and configure the Terraform CLI.
  • Authentication setup: Configure CLI authentication using API tokens or OAuth, ensuring HCP Terraform CLI connection to your environment with proper permissions.
  • Integration: Integrate with your existing CI/CD pipelines.
  • Choose starting point: Initiate from the UI or command line.
    Maintenance Tasks

  • Terraform version updates: Manage updates for Terraform versions on local machines or CI servers.
  • Branch management: Maintain disciplined branching and versioning of Terraform configuration files locally or through CI/CD pipelines.
    Security Concerns

  • Administrators must manage credentials like HCP Terraform API tokens and apply access controls on configuration files.
    Architectural and Process Complexity

  • Utilizes the Terraform binary and is familiar to those experienced with Terraform Community Edition.
  • Requires more effort to integrate into CI/CD compared to VCS-driven workflow.

    API

    The API-driven run workflow offers greater flexibility than the other two workflows but often requires custom tooling. It relies on an orchestrator capable of monitoring changes to Terraform code and making calls to HCP Terraform to initiate runs. This orchestration can be automated through a CI/CD system or other tools capable of monitoring changes and responding accordingly.

    Custom tooling is often created by wrapping the HCP Terraform API to perform specific actions based on outputs or conditions. For example, post-Terraform runs, a custom wrapper might trigger tools like Ansible for further actions. This approach is commonly adopted to meet organizational needs, address specific requirements, or handle custom workflows that standardized solutions cannot fully support.

    Due to its complexity, this workflow has the steepest learning curve. Acquaint yourself with the HCP Terraform API before adopting this workflow. Workflow authors should have expertise in CI/CD integrations and scalable practices.

    Use This When

  • You need a custom workflow integrated with your specific toolchain.
  • You require highly granular controls for each workflow step.
  • You want workflows extending beyond Terraform provisioning.
  • You aim for low-touch, heavily automated workflows.
  • You prefer not to grant direct access to HCP Terraform UI or CLI, using a different central platform for tool integration.
    Setup Tasks

  • Requirements: Choose your orchestration tool (e.g., CI/CD system) for HCP Terraform automation and explore integration points.
  • Authentication setup: Set up API tokens for HCP Terraform authentication.
  • Automation scripts: Develop scripts interacting with the Terraform API, possibly using tools like curl, Postman, or scripting languages like Python or Go.
  • Webhook configuration: Set up webhooks to trigger automated workflows based on specific events.
  • Integration: Build a wrapper for workflow integration with any tools or systems.
    Maintenance Tasks

  • API changes and deprecation: Continuously monitor API updates or deprecations to ensure script compatibility.
  • Run tracking: Use API capabilities to monitor and track run progress.
  • Error handling and logging: Implement logging and alerting mechanisms for real-time issue detection and troubleshooting.
    Security Concerns

  • Credential management: Administrators must manage credentials, including HCP Terraform API tokens.
  • Granular access control: Implement proper policies like OAuth scopes to limit infrastructure exposure.
  • API endpoint security: Ensure all API calls occur over HTTPS to prevent MITM attacks.
  • Security strategy: Establish and manage authentication, authorization, monitoring, and anomaly detection.
    Architectural and Process Complexity

  • Complex setup, maintenance, and debugging, especially with multiple system integrations.

    Workflows Change and Evolve

    In real-world infrastructure development, workflows can vary depending on the project’s stage and may shift as the project evolves. Consider the project’s goals, timeline, maintenance costs, and team skill levels to determine the most suitable workflow for your current state.

    Summary of Workflows

    Each organization’s journey with infrastructure provisioning is unique, varying at organizational, team, and individual levels. Here’s a quick reference for the three HCP Terraform workflows:

  • UI/VCS
    • Setup effort: Low
    • Maintenance effort: Low
    • Security concerns: Access to VCS and HCP Terraform
    • Architectural and process complexity: Low
    • Flexibility for customization: Low
  • CLI
    • Setup effort: Moderate
    • Maintenance effort: Moderate
    • Security concerns: Credentials and configuration files
    • Architectural and process complexity: Moderate
    • Flexibility for customization: Moderate
  • API
    • Setup effort: High
    • Maintenance effort: High
    • Security concerns: Credentials and API endpoints
    • Architectural and process complexity: High
    • Flexibility for customization: High

      Additional References

      For more detailed information, you can explore the HCP Terraform blog, further delving into the benefits and challenges of each workflow.

      In conclusion, selecting the appropriate HCP Terraform run workflow requires a thorough understanding of your organization’s specific needs, resources, and expertise. By weighing the various factors and considering the unique requirements of your team, you can optimize your infrastructure provisioning processes effectively.

For more Information, Refer to this article.

Neil S
Neil S
Neil is a highly qualified Technical Writer with an M.Sc(IT) degree and an impressive range of IT and Support certifications including MCSE, CCNA, ACA(Adobe Certified Associates), and PG Dip (IT). With over 10 years of hands-on experience as an IT support engineer across Windows, Mac, iOS, and Linux Server platforms, Neil possesses the expertise to create comprehensive and user-friendly documentation that simplifies complex technical concepts for a wide audience.
Watch & Subscribe Our YouTube Channel
YouTube Subscribe Button

Latest From Hawkdive

You May like these Related Articles

LEAVE A REPLY

Please enter your comment!
Please enter your name here

This site uses Akismet to reduce spam. Learn how your comment data is processed.