Chapter 11: Cloud Execution: Google Cloud Batch & Azure CycleCloud

Chapter 11: Cloud Execution: Google Cloud Batch & Azure CycleCloud

In multi-cloud environments, organizations deploy Nextflow across Google Cloud Platform (GCP) and Microsoft Azure to avoid vendor lock-in and leverage specific cloud AI services.

1. Configuring Google Cloud Batch in Nextflow

plugins {
    id 'nf-google'
}

profiles {
    googlebatch {
        process {
            executor = 'google-batch'
        }

        google {
            project  = 'my-genomics-project-123'
            location = 'us-central1'
        }

        workDir = 'gs://my-genomics-bucket/work'
    }
}

2. Microsoft Azure Batch Configuration

plugins {
    id 'nf-azure'
}

profiles {
    azure {
        process {
            executor = 'azurebatch'
        }

        azure {
            batch {
                location = 'eastus'
                accountName = 'myazurebatchaccount'
                accountKey = secrets.AZURE_BATCH_KEY
                autoPoolMode = true
            }
            storage {
                accountName = 'mystorageaccount'
                accountKey = secrets.AZURE_STORAGE_KEY
            }
        }

        workDir = 'az://mycontainer/work'
    }
}