Manage GCS storage (V37.0+)

Coming in V37.0: Google Cloud storage support

Google Cloud Storage Credentials update procedure

How to get the client credentials?

1. Go to the google cloud Console.

2. Navigate to IAM & Admin - In the left sidebar, go to IAM & Admin > Service Accounts.

1. Create a New Service Account: Click on the "Create Service Account" button at the top.

1. Fill in the Service Account Details:

   • Service account name: Give your service account a name.

   • Service account ID: This will be automatically generated based on the name.

   • Description: Optionally, provide a description for the service account.

   Click "Create and Continue".

   example:

1. Assign Roles to the Service Account:

   • In the Grant this service account access to project step, you’ll assign the necessary roles.

   • Grant these role:

     • "storage object viewer" (read-only access)

2. Create the Service Account:

   • After assigning the roles, click "Done".

3. Generate and Download a Key:

   • Find your newly created service account, click the three dots on the right, and select "Manage Keys".

   • Click Add Key > Create New Key and choose the JSON format.

   • Download the key and store it securely, as it is used for authentication in your code or applications.

4. Encode the key in base 64:

   • use python function: put this function and your json (here named json_file.json) in the same directory and run.\

     import json
     import base64
     
     
     def encode_json_to_base64(json_file):
         # Read JSON data from file
         with open(json_file, 'r') as file:
             json_data = json.load(file)
     
         # Convert the JSON data to a string
         json_str = json.dumps(json_data)
     
         # Encode the string to bytes, then to Base64
         json_bytes = json_str.encode('utf-8')
         base64_bytes = base64.b64encode(json_bytes)
     
         # Convert Base64 bytes back to a string
         base64_str = base64_bytes.decode('utf-8')
         # Print the Base64-encoded string
         print(base64_str)
     
     
     encode_json_to_base64('json_file.json')

   • save the output printed.

Add the storage provider to Emedgene platform:

• Add the above 3 values into the appropriate fields:

  • Client_credentials_base64: pasting the output of 8.

  • Bucket: the bucket name.

  • Path: for default, fill with / else, put your path in the bucket. Seperate directories with /

CORS - Visualisation

1. Download and install the Google Cloud SDK from the Google Cloud SDK Install page. LINK
2. Select Your Platform (Windows, macOS, or Linux), download and run.

3. Initialize and Authenticate with Google Cloud: In the Cloud SDK Shell/terminal, run: gcloud init This will open a browser window to authenticate your Google account. Follow the instructions to log in and select your project.

4. Set CORS Configuration via gcloud: Create a JSON file (cors.json) on your machine with the CORS rules. Example\ it should look like:

   [
       {
         "origin": ["https://<host_name>.emg.illumina.com"],
         "method": ["GET"],
         "responseHeader": ["emgauthorization"],
         "maxAgeSeconds": 3600
       }
   ]

notice:

• origin: if using Illumina cloud:

  https://host_name.emg.illumina.com

  else, Emedgene cloud:

  https://host_name.emedgene.com

1. Apply CORS Configuration to Your Bucket: run the next command. gcloud storage buckets update gs://your-bucket-name --cors-file=cors.json

2. Verify the CORS Configuration: gcloud storage buckets describe gs://your-bucket-name