- 29 Mar, 2016 1 commit
-
-
dsansome@chromium.org authored
Fixes this flake: https://build.chromium.org/p/chromium.infra/builders/infra-continuous-mac-10.9-64/builds/1729/steps/upload%20go%20bin/logs/stdio BUG= Review URL: https://codereview.chromium.org/1824223002 git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@299525 0039d316-1c4b-4281-b951-d872f2087c98
-
- 26 Feb, 2016 1 commit
-
-
hinoka@chromium.org authored
This doesn't actually look like it's used. BUG= Review URL: https://codereview.chromium.org/1747443002 git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@298997 0039d316-1c4b-4281-b951-d872f2087c98
-
- 25 Sep, 2015 1 commit
-
-
hinoka@chromium.org authored
BUG=504452 Review URL: https://codereview.chromium.org/1223793008 git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@296882 0039d316-1c4b-4281-b951-d872f2087c98
-
- 17 Sep, 2015 1 commit
-
-
hinoka@chromium.org authored
BUG= Review URL: https://codereview.chromium.org/1352043004 git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@296750 0039d316-1c4b-4281-b951-d872f2087c98
-
- 18 Aug, 2015 1 commit
-
-
ricow@chromium.org authored
Reland 0c7d94eb: Add support for tar.gz archive files to download from download_from_google_storage I fixed the overlapping argument and fixed up the tests so that the actual sha1 hash is now passed (since the tool actually check now that the downloaded file hash matches) BUG= R=hinoka@google.com Review URL: https://codereview.chromium.org/1285423002 . git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@296346 0039d316-1c4b-4281-b951-d872f2087c98
-
- 29 Jul, 2015 1 commit
-
-
hinoka@chromium.org authored
This does one last check to see if the file downloaded by download_from_google_storage.py actually matches its sha1 BUG= Review URL: https://codereview.chromium.org/1252313005 git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@296155 0039d316-1c4b-4281-b951-d872f2087c98
-
- 30 Jun, 2015 2 commits
-
-
boliu@chromium.org authored
Revert of Add support for tar.gz archive files to download from download_from_google_storage (patchset #8 id:140001 of https://codereview.chromium.org/807463005/) Reason for revert: This conflicts with (and duplicates?) the -z option below and has broken one of the chrome android downstream bots. Safer to revert than trying to fix Original issue's description: > Add support for tar.gz archive files to download from download_from_google_storage > > Also, add support for directly setting the public read flag on uploaded files. > > The support for tar.gz allows the uploaded bundle to be a single tar.gz file instead of a lot of individual files using the directory support already in. The benefit here is that it is much easier to update the dependency. Simply clean out the existing files, copy in the new ones, create a tar.gz file, with the same name as the directory + 'tar.gz'. If the directory name and file name does not match up we will not clean up existing artifacts on download (i.e., there can be left over files after extracting). > > I am doing this because I am moving a bunch of the dart dependencies to gcs, and a lot of our dependencies is much easier to manage with this in. If you don't like this, I can simply wrap the download script in another python script and do the logic there, but this may be handy for other people as well. > > Committed: http://src.chromium.org/viewvc/chrome?view=rev&revision=295872 TBR=hinoka@google.com,hinoka@chromium.org,azarchs@chromium.org,ricow@google.com NOPRESUBMIT=true NOTREECHECKS=true NOTRY=true Review URL: https://codereview.chromium.org/1209033006 git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@295883 0039d316-1c4b-4281-b951-d872f2087c98
-
ricow@google.com authored
Also, add support for directly setting the public read flag on uploaded files. The support for tar.gz allows the uploaded bundle to be a single tar.gz file instead of a lot of individual files using the directory support already in. The benefit here is that it is much easier to update the dependency. Simply clean out the existing files, copy in the new ones, create a tar.gz file, with the same name as the directory + 'tar.gz'. If the directory name and file name does not match up we will not clean up existing artifacts on download (i.e., there can be left over files after extracting). I am doing this because I am moving a bunch of the dart dependencies to gcs, and a lot of our dependencies is much easier to manage with this in. If you don't like this, I can simply wrap the download script in another python script and do the logic there, but this may be handy for other people as well. Review URL: https://codereview.chromium.org/807463005 git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@295872 0039d316-1c4b-4281-b951-d872f2087c98
-
- 17 Jun, 2015 1 commit
-
-
hinoka@chromium.org authored
This does two noticable things: * Prints a message when "download_from_google_storage --config" is run to tell the user to enter "0" for the project ID prompt * Removes the ".boto.depot_tools" boto file and defaults the boto file to grant fullcontrol scopes. Context: We restricted the depot_tools specific scopes to be readonly out of concern that we would be forcing every developer to hold a set of non-expiring write access credentials on their workstation. But this distinction has caused a great deal of pain and anguish with confusing credentials (who would've thought ~/.boto.depot_tools would exist and might be broken?), and not for huge security gains. Most people don't have write access to buckets, and the ones that do definitely has a fullcontrol boto file already. BUG= Review URL: https://codereview.chromium.org/1182583003 git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@295728 0039d316-1c4b-4281-b951-d872f2087c98
-
- 26 May, 2015 1 commit
-
-
nparker@chromium.org authored
BUG= Review URL: https://codereview.chromium.org/1091373004 git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@295416 0039d316-1c4b-4281-b951-d872f2087c98
-
- 17 Feb, 2015 1 commit
-
-
primiano@chromium.org authored
Currently check_bucket_permissions() in download_from_google_storage.py performs a gsutil ls gs://bucket to determine whether the user has access to the bucket or not. This can be an EXTREMELY expensive operation (~minutes) if the bucket in question has a lot of objects in the root (real case: chrome-telemetry). It is worth noting that check_bucket_permissions() is not called just for uploads but also for downloads, hence this is slowing down all invocations of gclient sync on users and bots machine. Removing the check_bucket_permissions() and let gsutil fail with an Unauthorized error message if ACLs are not met. BUG=458059 Review URL: https://codereview.chromium.org/923473002 git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@294083 0039d316-1c4b-4281-b951-d872f2087c98
-
- 15 Jan, 2015 1 commit
-
-
mmoss@chromium.org authored
This prevents --no_auth from always clearing BOTO_CONFIG, since there are times when a BOTO is needed for other things than just auth info (e.g. proxy settings). BUG=443523 R=hinoka@chromium.org, szager@chromium.org Review URL: https://codereview.chromium.org/844373002 git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@293652 0039d316-1c4b-4281-b951-d872f2087c98
-
- 17 Dec, 2014 2 commits
-
-
dpranke@chromium.org authored
The new GSUtil (or gs protocol, who knows) strips off the redundent x-goog-meta string from the metadata key. This CL compensates for that. Also since we're on 4.7, we can use the faster gsutil stat instead of gsutil ls -L. BUG= TEST=ran download_from_google_storage against compiler_proxy.sha, works NOTREECHECKS=true NOTRY=true R=dnj@chromium.org, pgervais@chromium.org Review URL: https://codereview.chromium.org/809123003 git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@293432 0039d316-1c4b-4281-b951-d872f2087c98
-
hinoka@chromium.org authored
This pins gsutil to a vanilla 4.7 instead of the weird custom 3.4 we have in depot_tools BUG= R=pgervais@chromium.org Review URL: https://codereview.chromium.org/797663003 git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@293413 0039d316-1c4b-4281-b951-d872f2087c98
-
- 04 Dec, 2014 4 commits
-
-
ojan@chromium.org authored
check_bucket_permissions should no longer return a tuple. TBR=vadimsh@chromium.org Review URL: https://codereview.chromium.org/759013007 git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@293255 0039d316-1c4b-4281-b951-d872f2087c98
-
ojan@chromium.org authored
check_bucket_permissions() takes exactly 2 arguments (3 given) TBR=vadimsh@chromium.org Review URL: https://codereview.chromium.org/780113003 git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@293254 0039d316-1c4b-4281-b951-d872f2087c98
-
ojan@chromium.org authored
--sha1_filename unnecessarily forces the output to have the same filename and be in the same directory. The code in main already correctly sets the file name to the sha1_filename minus the .sha1, so the only change is to actually use the --output path the same way the rest of the code does. R=iannucci@chromium.org Review URL: https://codereview.chromium.org/752803003 git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@293251 0039d316-1c4b-4281-b951-d872f2087c98
-
ojan@chromium.org authored
Checking bucket permissions takes ~400ms. We don't need to do this if --no-auth because we know we won't get a 403 and the 404 check will be handled later when we try to actually download the file. Also, remove the check for a null bucket. This can't happen since we will throw a parser error in the main function before we get to this code. R=iannucci@chromium.org Review URL: https://codereview.chromium.org/772203002 git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@293250 0039d316-1c4b-4281-b951-d872f2087c98
-
- 04 Nov, 2014 1 commit
-
-
hinoka@chromium.org authored
BUG= Review URL: https://codereview.chromium.org/696023002 git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@292819 0039d316-1c4b-4281-b951-d872f2087c98
-
- 17 Jan, 2014 1 commit
-
-
hinoka@google.com authored
A very common pattern for the users of this script is to have binaries in three different folders, eg: some/folder/win/bin.exe some/folder/mac/bin some/folder/linux/bin Instead of using --platform to specify three different paths, we can recognize this usage and pass in --auto_platform, and then --directory some/folder/ as the target. When enumerating files, it will match each individual file to see if they have (win|mac|linux) as a parent folder name, and filter out non-matching platforms. BUG= TEST= src/third_party/clang_format/bin$ download_from_google_storage --auto_platform --no_auth -b chromium-clang-fomat -dr . 0> Downloading ./linux/clang-format... Review URL: https://codereview.chromium.org/126893002 git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@245643 0039d316-1c4b-4281-b951-d872f2087c98
-
- 10 Jan, 2014 1 commit
-
-
hinoka@google.com authored
When download_from_google_storage --config is run, it should be implied that the user just wants to download, not upload. This change passes the '-r' flag into gsutil config, which requests a read-only scoped token rather than write token. This is saved in ~/.boto.depot_tools so that it doesn't conflict with a ~/.boto file crated later that may have write permissions. BUG= Review URL: https://codereview.chromium.org/135153002 git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@244276 0039d316-1c4b-4281-b951-d872f2087c98
-
- 12 Dec, 2013 1 commit
-
-
hinoka@chromium.org authored
Code path: 1. plugins.sso_auth is imported, which adds the AuthHandler class to the global state. 2. HasConfiguredCredentials() in gslib/utils.py is called by gsutil, and will return true if "prodaccess" exists on the system, which tells the system that we don't want a no-op auth handler. 3. When a command is called, all the auth handlers are cycled through and sso_auth.SSOAuth is called, which calls a stubby command to emit a gaiamint'ed oauth2 access token, which is then used as the Authorization Header if --bypass_prodaccess is passed in, then: 1. HasConfiguredCredentials() will bypass the check for prodaccess, as if it didn't exist. 2. plugins.sso_auth does not get imported. Which will essentially cause gsutil to behave as if this patch never existed. So the expected behavior is: =.boto file does not exist, prodaccess exists, but unauthenticated= Failure: No handler was ready to authenticate. 3 handlers were checked. ['OAuth2Auth', 'HmacAuthV1Handler', 'SSOAuth'] Check your credentials. =.boto file exists, prodaccess exists, but unauthenticated= sso_auth will raise NotReadyToAuthenticate, and the .boto file will be used instead =.boto file exists, prodaccess exists, authenticated= sso_auth will be run _after_ the default gsutil authenticator, which causes the sso_auth to be used over whatever the default authentication is. bypass_prodaccess is passed in by default to upload_to_google_storage because we expect people who use upload_to_google_storage to not need prodaccess and have their own boto file already. Also the sso_auth plugin will only request a readonlyi token, which will not work for uploading. BUG=258152 Review URL: https://codereview.chromium.org/86123002 git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@240266 0039d316-1c4b-4281-b951-d872f2087c98
-
- 06 Dec, 2013 1 commit
-
-
brettw@chromium.org authored
This should make Cygwin work to match "win32". Review URL: https://codereview.chromium.org/108743002 git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@239244 0039d316-1c4b-4281-b951-d872f2087c98
-
- 05 Dec, 2013 2 commits
-
-
brettw@chromium.org authored
Review URL: https://codereview.chromium.org/107323002 git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@239057 0039d316-1c4b-4281-b951-d872f2087c98
-
brettw@chromium.org authored
This normalizes sys.platform so people running cygwin are treated like Windows. Review URL: https://codereview.chromium.org/104763003 git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@238999 0039d316-1c4b-4281-b951-d872f2087c98
-
- 25 Nov, 2013 1 commit
-
-
scottmg@chromium.org authored
Seems overly verbose when run in "gclient runhooks". R=iannucci@chromium.org, brettw@chromium.org Review URL: https://codereview.chromium.org/58783002 git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@237120 0039d316-1c4b-4281-b951-d872f2087c98
-
- 19 Nov, 2013 1 commit
-
-
hinoka@chromium.org authored
BUG=321254 Review URL: https://codereview.chromium.org/76583002 git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@236039 0039d316-1c4b-4281-b951-d872f2087c98
-
- 28 Oct, 2013 1 commit
-
-
brettw@chromium.org authored
Adds a command line flag --platform to control which platforms the given download occurs on. Automatically marks downloads executable on non-Windows if the header x-goog-meta-executable is set. Automatically set this flag on upload when the executable bit is set. BUG=233100 Review URL: https://codereview.chromium.org/42273002 git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@231387 0039d316-1c4b-4281-b951-d872f2087c98
-
- 19 Sep, 2013 1 commit
-
-
hinoka@google.com authored
We appear to be getting 403's when encountering errors other than mismatched credentials. The error message was originally hidden to avoid clutter, but it has become obvious that there are other transient errors hidden behind a 403. This change prints the original gsutil error message regardless of whether or not we enconter a 403 or 404. BUG= Review URL: https://chromiumcodereview.appspot.com/23484061 git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@224195 0039d316-1c4b-4281-b951-d872f2087c98
-
- 28 Jun, 2013 1 commit
-
-
petermayo@chromium.org authored
Able to pull internal tools, but distinct from the source that those tools may eventually build. TBR=cmp@chromium.org BUG=252226 TEST=local buildslave Review URL: https://chromiumcodereview.appspot.com/17351008 git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@209166 0039d316-1c4b-4281-b951-d872f2087c98
-
- 26 Jun, 2013 1 commit
-
-
hinoka@chromium.org authored
In order to support both developer workflow and not breaking bots, if the script fails on a 403 in a bucket, it'll print a message asking developers to run "download_from_google_storage --config" in order to create a new boto file. This is not done automatically because it would break bots (Imagine hitting a 403, and then gsutil wiping the .boto file, waiting for input, then dying). BUG=231699,176331 Review URL: https://chromiumcodereview.appspot.com/17590010 git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@208806 0039d316-1c4b-4281-b951-d872f2087c98
-
- 20 Jun, 2013 1 commit
-
-
petermayo@chromium.org authored
It appears to be parsed and ignored, seems like a bug to me. BUG=None TEST=local/manual. Review URL: https://chromiumcodereview.appspot.com/17265012 git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@207404 0039d316-1c4b-4281-b951-d872f2087c98
-
- 04 Jun, 2013 1 commit
-
-
hinoka@google.com authored
When a developer runs download_from_google_storage, and they don't have a .boto file, the tool automatically runs "gsutil config" to create one for them. Unfortunately, a side consequence is that if a bot runs the script, and it has a boto file that 403's, then it would run "gsutil config" which moves the .boto to .boto.bak, and creates an empty .boto file. This should not be the intended action. This CL changes so that "gsutil config" is not called, but instead just fails with a message telling the dev to run that command. TBR=maruel@chromium.org Review URL: https://codereview.chromium.org/16072023 git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@203824 0039d316-1c4b-4281-b951-d872f2087c98
-
- 13 Mar, 2013 1 commit
-
-
hinoka@google.com authored
continuation of: https://codereview.chromium.org/11664024 Moved it from chrome/trunk/src/build to depot_tools/ BUG=153360 TEST=two unittests included in tests/ For end-to-end testing, check out a large directory. Run find . -name .svn -prune -o -size +1000k -type f -print0 | upload_to_google_storage.py -b chrome-artifacts -0 - (replacing chrome-artifacts with an upload-able bucket) to test upload run "find . -name .svn -prune -o -size +1000k -type f -print0 | xargs -0 rm" to remove the files uploaded. Check that the large binary files have been removed run "download_from_google_storage.py -r -d -b chrome-artifacts ." to download the files again. Review URL: https://chromiumcodereview.appspot.com/12042069 git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@187951 0039d316-1c4b-4281-b951-d872f2087c98
-