For the past one and half months, I was a member of an AI development team. Our team collaborated and trained ML models on GCP, and finally won a prize in an AI competition held by ITRI (Industrial Technology Research Institute) and MOL (Ministry of Labor) last week. Therefore, I will share a series of GCP tips that we found very useful with you!
As a team, we needed to check and align every one's progress, so we chose Google Drive as the only place to upload all the files, documents and raw materials. (One of the reasons: Every member has some experience using Google Drive, but not GCP). To make it easier, we synced the files between Google Drive and GCP by using simple codes in Google Colab with just 3 steps:
Connect Colab to Google Drive: from google.colab import drive drive.flush_and_unmount() drive.mount('/content/drive/', force_remount=True) In this step, you will need to login Google Drive account and copy authentication key back to Colab. Then, you will get connected and see all your Drive folders and files.
Connect Colab to GCP: from google.colab import auth auth.authenticate_user() project_id = 'YOUR-PROJECT_ID' !gcloud config set project {project_id} !gsutil ls In this step, you will need to login GCP account and copy authentication key back to Colab. Then, you will get connected and see all your buckets.
Copy assigned Google Drive files to Cloud Storage on GCP: bucket_name = 'YOUR-STORAGE-BUCKET-NAME' !gsutil -m cp -r /content/drive/MyDrive/$YOUR-DRIVE-PATH gs://$YOUR-GCP-BUCKET-PATH After you finish filling in your own info and run it, you can see the assigned files are copying from Google Drive to GCP. Wait a few minutes, ALL DONE!
Message or email me if you have any further questions. I will be happy to share my experience with you!
Comments