This is needed for https://chromiumcodereview.appspot.com/12042069/

Which uses gsutil to download objects from Google Storage based on SHA1 sums 

Continuation of: https://chromiumcodereview.appspot.com/12317103/ 
Rietveld didn't like a giant CL with all of gsutil (kept crashing on upload), 
The CL is being split into three parts 

Related: 
https://chromiumcodereview.appspot.com/12685010/ (gsutil/gslib) 
https://codereview.chromium.org/12685009/ (gsutil/) 

Review URL: https://chromiumcodereview.appspot.com/12633019

git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@188841 0039d316-1c4b-4281-b951-d872f2087c98

third_party/boto/LICENSE

+Permission is hereby granted, free of charge, to any person obtaining a
+copy of this software and associated documentation files (the
+"Software"), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish, dis-
+tribute, sublicense, and/or sell copies of the Software, and to permit
+persons to whom the Software is furnished to do so, subject to the fol-
+lowing conditions:
+
+The above copyright notice and this permission notice shall be included
+in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS
+OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABIL-
+ITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT
+SHALL THE AUTHOR BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
+WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+IN THE SOFTWARE.

third_party/boto/README.chromium

+Name: boto 
+URL: https://github.com/boto/boto 
+Version: 2.8.0 
+License: MIT 
+
+Description:
+Python interface to Amazon Web Services.
+
+Modifications:
+* Removed boto/docs/
+* Removed boto/tests/
+* Removed boto/boto/setup.py
+* Removed boto/boto/pylintrc
+* Removed boto/boto/ec2
+* Removed boto/boto/beanstalk
+* Removed boto/boto/cloudsearch
+* Removed boto/boto/dynamodb
+* Removed boto/boto/swf
+* Removed boto/boto/rds
+* Removed boto/boto/sdb
+* Removed boto/boto/vpc
+* Removed boto/boto/route53
+* Removed boto/boto/iam
+* Removed boto/boto/sns
+* Removed boto/boto/cloudformations
+* Removed boto/boto/elasticache
+* Removed boto/boto/elastictranscoder
+* Removed boto/boto/mturk
+* Removed boto/boto/cloudfront
+* Removed boto/boto/sqs
+* Removed boto/boto/mws
+* Removed boto/boto/emr
+* Removed boto/boto/pyami/installers
+* Removed boto/bin
+* Removed boto/.gitignore
+* Removed boto/.travis.yml
+* Removed boto/Changelog.rst
+* Removed boto/MANIFEST.in
+* Moved boto down to be a first level directory
+
+Full license is in the LICENSE file.

third_party/boto/README.rst

+####
+boto
+####
+boto 2.8.0
+31-Jan-2013
+
+.. image:: https://secure.travis-ci.org/boto/boto.png?branch=develop
+        :target: https://secure.travis-ci.org/boto/boto
+
+************
+Introduction
+************
+
+Boto is a Python package that provides interfaces to Amazon Web Services.
+At the moment, boto supports:
+
+* Compute
+
+  * Amazon Elastic Compute Cloud (EC2)
+  * Amazon Elastic Map Reduce (EMR)
+  * AutoScaling
+
+* Content Delivery
+
+  * Amazon CloudFront
+
+* Database
+
+  * Amazon Relational Data Service (RDS)
+  * Amazon DynamoDB
+  * Amazon SimpleDB
+  * Amazon ElastiCache
+
+* Deployment and Management
+
+  * AWS Elastic Beanstalk
+  * AWS CloudFormation
+  * AWS Data Pipeline
+
+* Identity & Access
+
+  * AWS Identity and Access Management (IAM)
+
+* Application Services
+
+  * Amazon CloudSearch
+  * Amazon Simple Workflow Service (SWF)
+  * Amazon Simple Queue Service (SQS)
+  * Amazon Simple Notification Server (SNS)
+  * Amazon Simple Email Service (SES)
+
+* Montoring
+
+  * Amazon CloudWatch
+
+* Networking
+
+  * Amazon Route53
+  * Amazon Virtual Private Cloud (VPC)
+  * Elastic Load Balancing (ELB)
+
+* Payments and Billing
+
+  * Amazon Flexible Payment Service (FPS)
+
+* Storage
+
+  * Amazon Simple Storage Service (S3)
+  * Amazon Glacier
+  * Amazon Elastic Block Store (EBS)
+  * Google Cloud Storage
+
+* Workforce
+
+  * Amazon Mechanical Turk
+
+* Other
+
+  * Marketplace Web Services
+
+The goal of boto is to support the full breadth and depth of Amazon
+Web Services.  In addition, boto provides support for other public
+services such as Google Storage in addition to private cloud systems
+like Eucalyptus, OpenStack and Open Nebula.
+
+Boto is developed mainly using Python 2.6.6 and Python 2.7.1 on Mac OSX
+and Ubuntu Maverick.  It is known to work on other Linux distributions
+and on Windows.  Boto requires no additional libraries or packages
+other than those that are distributed with Python.  Efforts are made
+to keep boto compatible with Python 2.5.x but no guarantees are made.
+
+************
+Installation
+************
+
+Install via `pip`_:
+
+::
+
+	$ pip install boto
+
+Install from source:
+
+::
+
+	$ git clone git://github.com/boto/boto.git
+	$ cd boto
+	$ python setup.py install
+
+**********
+ChangeLogs
+**********
+
+To see what has changed over time in boto, you can check out the
+`release notes`_ in the wiki.
+
+***************************
+Finding Out More About Boto
+***************************
+
+The main source code repository for boto can be found on `github.com`_.
+The boto project uses the `gitflow`_ model for branching.
+
+`Online documentation`_ is also available. The online documentation includes
+full API documentation as well as Getting Started Guides for many of the boto
+modules.
+
+Boto releases can be found on the `Python Cheese Shop`_.
+
+Join our IRC channel `#boto` on FreeNode.
+Webchat IRC channel: http://webchat.freenode.net/?channels=boto
+
+Join the `boto-users Google Group`_.
+
+*************************
+Getting Started with Boto
+*************************
+
+Your credentials can be passed into the methods that create
+connections.  Alternatively, boto will check for the existance of the
+following environment variables to ascertain your credentials:
+
+**AWS_ACCESS_KEY_ID** - Your AWS Access Key ID
+
+**AWS_SECRET_ACCESS_KEY** - Your AWS Secret Access Key
+
+Credentials and other boto-related settings can also be stored in a
+boto config file.  See `this`_ for details.
+
+Copyright (c) 2006-2012 Mitch Garnaat <mitch@garnaat.com>
+Copyright (c) 2010-2011, Eucalyptus Systems, Inc.
+Copyright (c) 2012 Amazon.com, Inc. or its affiliates.
+All rights reserved.
+
+.. _pip: http://www.pip-installer.org/
+.. _release notes: https://github.com/boto/boto/wiki
+.. _github.com: http://github.com/boto/boto
+.. _Online documentation: http://docs.pythonboto.org
+.. _Python Cheese Shop: http://pypi.python.org/pypi/boto
+.. _this: http://code.google.com/p/boto/wiki/BotoConfig
+.. _gitflow: http://nvie.com/posts/a-successful-git-branching-model/
+.. _neo: https://github.com/boto/boto/tree/neo
+.. _boto-users Google Group: https://groups.google.com/forum/?fromgroups#!forum/boto-users

third_party/boto/auth_handler.py

+# Copyright 2010 Google Inc.
+#
+# Permission is hereby granted, free of charge, to any person obtaining a
+# copy of this software and associated documentation files (the
+# "Software"), to deal in the Software without restriction, including
+# without limitation the rights to use, copy, modify, merge, publish, dis-
+# tribute, sublicense, and/or sell copies of the Software, and to permit
+# persons to whom the Software is furnished to do so, subject to the fol-
+# lowing conditions:
+#
+# The above copyright notice and this permission notice shall be included
+# in all copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS
+# OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABIL-
+# ITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT
+# SHALL THE AUTHOR BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, 
+# WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+# IN THE SOFTWARE.
+
+"""
+Defines an interface which all Auth handlers need to implement.
+"""
+
+from plugin import Plugin
+
+class NotReadyToAuthenticate(Exception):
+  pass
+
+class AuthHandler(Plugin):
+
+    capability = []
+
+    def __init__(self, host, config, provider):
+        """Constructs the handlers.
+        :type host: string
+        :param host: The host to which the request is being sent.
+
+        :type config: boto.pyami.Config 
+        :param config: Boto configuration.
+
+        :type provider: boto.provider.Provider  
+        :param provider: Provider details.
+
+        Raises:
+            NotReadyToAuthenticate: if this handler is not willing to
+                authenticate for the given provider and config.
+        """
+        pass
+
+    def add_auth(self, http_request):
+        """Invoked to add authentication details to request.
+
+        :type http_request: boto.connection.HTTPRequest
+        :param http_request: HTTP request that needs to be authenticated.
+        """
+        pass

third_party/boto/cacerts/__init__.py

+# Copyright 2010 Google Inc.
+# All rights reserved.
+#
+# Permission is hereby granted, free of charge, to any person obtaining a
+# copy of this software and associated documentation files (the
+# "Software"), to deal in the Software without restriction, including
+# without limitation the rights to use, copy, modify, merge, publish, dis-
+# tribute, sublicense, and/or sell copies of the Software, and to permit
+# persons to whom the Software is furnished to do so, subject to the fol-
+# lowing conditions:
+#
+# The above copyright notice and this permission notice shall be included
+# in all copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS
+# OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABIL-
+# ITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT
+# SHALL THE AUTHOR BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
+# WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+# IN THE SOFTWARE.
+#

third_party/boto/compat.py

+# Copyright (c) 2012 Amazon.com, Inc. or its affiliates.  All Rights Reserved
+#
+# Permission is hereby granted, free of charge, to any person obtaining a
+# copy of this software and associated documentation files (the
+# "Software"), to deal in the Software without restriction, including
+# without limitation the rights to use, copy, modify, merge, publish, dis-
+# tribute, sublicense, and/or sell copies of the Software, and to permit
+# persons to whom the Software is furnished to do so, subject to the fol-
+# lowing conditions:
+#
+# The above copyright notice and this permission notice shall be included
+# in all copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS
+# OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABIL-
+# ITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT
+# SHALL THE AUTHOR BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
+# WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+# IN THE SOFTWARE.
+#
+
+# This allows boto modules to say "from boto.compat import json".  This is
+# preferred so that all modules don't have to repeat this idiom.
+try:
+    import simplejson as json
+except ImportError:
+    import json

third_party/boto/contrib/__init__.py

+# Copyright (c) 2006,2007 Mitch Garnaat http://garnaat.org/
+#
+# Permission is hereby granted, free of charge, to any person obtaining a
+# copy of this software and associated documentation files (the
+# "Software"), to deal in the Software without restriction, including
+# without limitation the rights to use, copy, modify, merge, publish, dis-
+# tribute, sublicense, and/or sell copies of the Software, and to permit
+# persons to whom the Software is furnished to do so, subject to the fol-
+# lowing conditions:
+#
+# The above copyright notice and this permission notice shall be included
+# in all copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS
+# OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABIL-
+# ITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT
+# SHALL THE AUTHOR BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, 
+# WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+# IN THE SOFTWARE.
+#
+

third_party/boto/contrib/ymlmessage.py

+# Copyright (c) 2006,2007 Chris Moyer
+#
+# Permission is hereby granted, free of charge, to any person obtaining a
+# copy of this software and associated documentation files (the
+# "Software"), to deal in the Software without restriction, including
+# without limitation the rights to use, copy, modify, merge, publish, dis-
+# tribute, sublicense, and/or sell copies of the Software, and to permit
+# persons to whom the Software is furnished to do so, subject to the fol-
+# lowing conditions:
+#
+# The above copyright notice and this permission notice shall be included
+# in all copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS
+# OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABIL-
+# ITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT
+# SHALL THE AUTHOR BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, 
+# WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+# IN THE SOFTWARE.
+
+"""
+This module was contributed by Chris Moyer.  It provides a subclass of the
+SQS Message class that supports YAML as the body of the message.
+
+This module requires the yaml module.
+"""
+from boto.sqs.message import Message
+import yaml
+
+class YAMLMessage(Message):
+    """
+    The YAMLMessage class provides a YAML compatible message. Encoding and
+    decoding are handled automaticaly.
+
+    Access this message data like such:
+
+    m.data = [ 1, 2, 3]
+    m.data[0] # Returns 1
+
+    This depends on the PyYAML package
+    """
+
+    def __init__(self, queue=None, body='', xml_attrs=None):
+        self.data = None
+        Message.__init__(self, queue, body)
+
+    def set_body(self, body):
+        self.data = yaml.load(body)
+
+    def get_body(self):
+        return yaml.dump(self.data)

third_party/boto/core/README

+What's This All About?
+======================
+
+This directory contains the beginnings of what is hoped will be the
+new core of boto.  We want to move from using httplib to using
+requests.  We also want to offer full support for Python 2.6, 2.7, and
+3.x.  This is a pretty big change and will require some time to roll
+out but this module provides a starting point.
+
+What you will find in this module:
+
+* auth.py provides a SigV2 authentication packages as a args hook for requests.
+* credentials.py provides a way of finding AWS credentials (see below).
+* dictresponse.py provides a generic response handler that parses XML responses
+  and returns them as nested Python data structures.
+* service.py provides a simple example of a service that actually makes an EC2
+  request and returns a response.
+
+Credentials
+===========
+
+Credentials are being handled a bit differently here.  The following
+describes the order of search for credentials:
+
+1. If your local environment for has ACCESS_KEY and SECRET_KEY variables
+   defined, these will be used.
+
+2. If your local environment has AWS_CREDENTIAL_FILE defined, it is assumed
+   that it will be a config file with entries like this:
+
+   [default]
+   access_key = xxxxxxxxxxxxxxxx
+   sercret_key = xxxxxxxxxxxxxxxxxx
+
+   [test]
+   access_key = yyyyyyyyyyyyyy
+   secret_key = yyyyyyyyyyyyyyyyyy
+
+   Each section in the config file is called a persona and you can reference
+   a particular persona by name when instantiating a Service class.
+
+3. If a standard boto config file is found that contains credentials, those
+   will be used.
+
+4. If temporary credentials for an IAM Role are found in the instance
+   metadata of an EC2 instance, these credentials will be used.
+
+Trying Things Out
+=================
+To try this code out, cd to the directory containing the core module.
+
+    >>> import core.service
+    >>> s = core.service.Service()
+    >>> s.describe_instances()
+
+This code should return a Python data structure containing information
+about your currently running EC2 instances.  This example should run in
+Python 2.6.x, 2.7.x and Python 3.x.
\ No newline at end of file

third_party/boto/core/__init__.py

+# Copyright (c) 2012 Mitch Garnaat http://garnaat.org/
+# Copyright (c) 2012 Amazon.com, Inc. or its affiliates.
+# All Rights Reserved
+#
+# Permission is hereby granted, free of charge, to any person obtaining a
+# copy of this software and associated documentation files (the
+# "Software"), to deal in the Software without restriction, including
+# without limitation the rights to use, copy, modify, merge, publish, dis-
+# tribute, sublicense, and/or sell copies of the Software, and to permit
+# persons to whom the Software is furnished to do so, subject to the fol-
+# lowing conditions:
+#
+# The above copyright notice and this permission notice shall be included
+# in all copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS
+# OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABIL-
+# ITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT
+# SHALL THE AUTHOR BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
+# WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+# IN THE SOFTWARE.
+#

third_party/boto/core/auth.py

+# Copyright (c) 2012 Mitch Garnaat http://garnaat.org/
+# Copyright (c) 2012 Amazon.com, Inc. or its affiliates.
+# All Rights Reserved
+#
+# Permission is hereby granted, free of charge, to any person obtaining a
+# copy of this software and associated documentation files (the
+# "Software"), to deal in the Software without restriction, including
+# without limitation the rights to use, copy, modify, merge, publish, dis-
+# tribute, sublicense, and/or sell copies of the Software, and to permit
+# persons to whom the Software is furnished to do so, subject to the fol-
+# lowing conditions:
+#
+# The above copyright notice and this permission notice shall be included
+# in all copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS
+# OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABIL-
+# ITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT
+# SHALL THE AUTHOR BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
+# WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+# IN THE SOFTWARE.
+#
+import requests.packages.urllib3
+import hmac
+import base64
+from hashlib import sha256
+import sys
+import datetime
+
+try:
+    from urllib.parse import quote
+except ImportError:
+    from urllib import quote
+
+
+class SigV2Auth(object):
+    """
+    Sign an Query Signature V2 request.
+    """
+    def __init__(self, credentials, api_version=''):
+        self.credentials = credentials
+        self.api_version = api_version
+        self.hmac = hmac.new(self.credentials.secret_key.encode('utf-8'),
+                             digestmod=sha256)
+
+    def calc_signature(self, args):
+        scheme, host, port = requests.packages.urllib3.get_host(args['url'])
+        string_to_sign = '%s\n%s\n%s\n' % (args['method'], host, '/')
+        hmac = self.hmac.copy()
+        args['params']['SignatureMethod'] = 'HmacSHA256'
+        if self.credentials.token:
+            args['params']['SecurityToken'] = self.credentials.token
+        sorted_params = sorted(args['params'])
+        pairs = []
+        for key in sorted_params:
+            value = args['params'][key]
+            pairs.append(quote(key, safe='') + '=' +
+                         quote(value, safe='-_~'))
+        qs = '&'.join(pairs)
+        string_to_sign += qs
+        print('string_to_sign')
+        print(string_to_sign)
+        hmac.update(string_to_sign.encode('utf-8'))
+        b64 = base64.b64encode(hmac.digest()).strip().decode('utf-8')
+        return (qs, b64)
+
+    def add_auth(self, args):
+        args['params']['Action'] = 'DescribeInstances'
+        args['params']['AWSAccessKeyId'] = self.credentials.access_key
+        args['params']['SignatureVersion'] = '2'
+        args['params']['Timestamp'] = datetime.datetime.utcnow().isoformat()
+        args['params']['Version'] = self.api_version
+        qs, signature = self.calc_signature(args)
+        args['params']['Signature'] = signature
+        if args['method'] == 'POST':
+            args['data'] = args['params']
+            args['params'] = {}

third_party/boto/core/credentials.py

+# Copyright (c) 2012 Mitch Garnaat http://garnaat.org/
+# Copyright (c) 2012 Amazon.com, Inc. or its affiliates.
+# All Rights Reserved
+#
+# Permission is hereby granted, free of charge, to any person obtaining a
+# copy of this software and associated documentation files (the
+# "Software"), to deal in the Software without restriction, including
+# without limitation the rights to use, copy, modify, merge, publish, dis-
+# tribute, sublicense, and/or sell copies of the Software, and to permit
+# persons to whom the Software is furnished to do so, subject to the fol-
+# lowing conditions:
+#
+# The above copyright notice and this permission notice shall be included
+# in all copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS
+# OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABIL-
+# ITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT
+# SHALL THE AUTHOR BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
+# WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+# IN THE SOFTWARE.
+#
+import os
+from six.moves import configparser
+import requests
+import json
+
+
+class Credentials(object):
+    """
+    Holds the credentials needed to authenticate requests.  In addition
+    the Credential object knows how to search for credentials and how
+    to choose the right credentials when multiple credentials are found.
+    """
+
+    def __init__(self, access_key=None, secret_key=None, token=None):
+        self.access_key = access_key
+        self.secret_key = secret_key
+        self.token = token
+
+
+def _search_md(url='http://169.254.169.254/latest/meta-data/iam/'):
+    d = {}
+    try:
+        r = requests.get(url, timeout=.1)
+        if r.content:
+            fields = r.content.split('\n')
+            for field in fields:
+                if field.endswith('/'):
+                    d[field[0:-1]] = get_iam_role(url + field)
+                else:
+                    val = requests.get(url + field).content
+                    if val[0] == '{':
+                        val = json.loads(val)
+                    else:
+                        p = val.find('\n')
+                        if p > 0:
+                            val = r.content.split('\n')
+                    d[field] = val
+    except (requests.Timeout, requests.ConnectionError):
+        pass
+    return d
+
+
+def search_metadata(**kwargs):
+    credentials = None
+    metadata = _search_md()
+    # Assuming there's only one role on the instance profile.
+    if metadata:
+        metadata = metadata['iam']['security-credentials'].values()[0]
+        credentials = Credentials(metadata['AccessKeyId'],
+                                  metadata['SecretAccessKey'],
+                                  metadata['Token'])
+    return credentials
+
+
+def search_environment(**kwargs):
+    """
+    Search for credentials in explicit environment variables.
+    """
+    credentials = None
+    access_key = os.environ.get(kwargs['access_key_name'].upper(), None)
+    secret_key = os.environ.get(kwargs['secret_key_name'].upper(), None)
+    if access_key and secret_key:
+        credentials = Credentials(access_key, secret_key)
+    return credentials
+
+
+def search_file(**kwargs):
+    """
+    If the 'AWS_CREDENTIAL_FILE' environment variable exists, parse that
+    file for credentials.
+    """
+    credentials = None
+    if 'AWS_CREDENTIAL_FILE' in os.environ:
+        persona = kwargs.get('persona', 'default')
+        access_key_name = kwargs['access_key_name']
+        secret_key_name = kwargs['secret_key_name']
+        access_key = secret_key = None
+        path = os.getenv('AWS_CREDENTIAL_FILE')
+        path = os.path.expandvars(path)
+        path = os.path.expanduser(path)
+        cp = configparser.RawConfigParser()
+        cp.read(path)
+        if not cp.has_section(persona):
+            raise ValueError('Persona: %s not found' % persona)
+        if cp.has_option(persona, access_key_name):
+            access_key = cp.get(persona, access_key_name)
+        else:
+            access_key = None
+        if cp.has_option(persona, secret_key_name):
+            secret_key = cp.get(persona, secret_key_name)
+        else:
+            secret_key = None
+        if access_key and secret_key:
+            credentials = Credentials(access_key, secret_key)
+    return credentials
+
+
+def search_boto_config(**kwargs):
+    """
+    Look for credentials in boto config file.
+    """
+    credentials = access_key = secret_key = None
+    if 'BOTO_CONFIG' in os.environ:
+        paths = [os.environ['BOTO_CONFIG']]
+    else:
+        paths = ['/etc/boto.cfg', '~/.boto']
+    paths = [os.path.expandvars(p) for p in paths]
+    paths = [os.path.expanduser(p) for p in paths]
+    cp = configparser.RawConfigParser()
+    cp.read(paths)
+    if cp.has_section('Credentials'):
+        access_key = cp.get('Credentials', 'aws_access_key_id')
+        secret_key = cp.get('Credentials', 'aws_secret_access_key')
+    if access_key and secret_key:
+        credentials = Credentials(access_key, secret_key)
+    return credentials
+
+AllCredentialFunctions = [search_environment,
+                          search_file,
+                          search_boto_config,
+                          search_metadata]
+
+
+def get_credentials(persona='default'):
+    for cred_fn in AllCredentialFunctions:
+        credentials = cred_fn(persona=persona,
+                              access_key_name='access_key',
+                              secret_key_name='secret_key')
+        if credentials:
+            break
+    return credentials

third_party/boto/core/dictresponse.py

+# Copyright (c) 2012 Mitch Garnaat http://garnaat.org/
+# Copyright (c) 2012 Amazon.com, Inc. or its affiliates.
+# All Rights Reserved
+#
+# Permission is hereby granted, free of charge, to any person obtaining a
+# copy of this software and associated documentation files (the
+# "Software"), to deal in the Software without restriction, including
+# without limitation the rights to use, copy, modify, merge, publish, dis-
+# tribute, sublicense, and/or sell copies of the Software, and to permit
+# persons to whom the Software is furnished to do so, subject to the fol-
+# lowing conditions:
+#
+# The above copyright notice and this permission notice shall be included
+# in all copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS
+# OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABIL-
+# ITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT
+# SHALL THE AUTHOR BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
+# WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+# IN THE SOFTWARE.
+#
+
+import xml.sax
+
+
+def pythonize_name(name, sep='_'):
+    s = ''
+    if name[0].isupper:
+        s = name[0].lower()
+    for c in name[1:]:
+        if c.isupper():
+            s += sep + c.lower()
+        else:
+            s += c
+    return s
+
+
+class XmlHandler(xml.sax.ContentHandler):
+
+    def __init__(self, root_node, connection):
+        self.connection = connection
+        self.nodes = [('root', root_node)]
+        self.current_text = ''
+
+    def startElement(self, name, attrs):
+        self.current_text = ''
+        t = self.nodes[-1][1].startElement(name, attrs, self.connection)
+        if t != None:
+            if isinstance(t, tuple):
+                self.nodes.append(t)
+            else:
+                self.nodes.append((name, t))
+
+    def endElement(self, name):
+        self.nodes[-1][1].endElement(name, self.current_text, self.connection)
+        if self.nodes[-1][0] == name:
+            self.nodes.pop()
+        self.current_text = ''
+
+    def characters(self, content):
+        self.current_text += content
+
+    def parse(self, s):
+        xml.sax.parseString(s, self)
+
+
+class Element(dict):
+
+    def __init__(self, connection=None, element_name=None,
+                 stack=None, parent=None, list_marker=None,
+                 item_marker=None, pythonize_name=False):
+        dict.__init__(self)
+        self.connection = connection
+        self.element_name = element_name
+        self.list_marker = list_marker or ['Set']
+        self.item_marker = item_marker or ['member', 'item']
+        if stack is None:
+            self.stack = []
+        else:
+            self.stack = stack
+        self.pythonize_name = pythonize_name
+        self.parent = parent
+
+    def __getattr__(self, key):
+        if key in self:
+            return self[key]
+        for k in self:
+            e = self[k]
+            if isinstance(e, Element):
+                try:
+                    return getattr(e, key)
+                except AttributeError:
+                    pass
+        raise AttributeError
+
+    def get_name(self, name):
+        if self.pythonize_name:
+            name = pythonize_name(name)
+        return name
+
+    def startElement(self, name, attrs, connection):
+        self.stack.append(name)
+        for lm in self.list_marker:
+            if name.endswith(lm):
+                l = ListElement(self.connection, name, self.list_marker,
+                                self.item_marker, self.pythonize_name)
+                self[self.get_name(name)] = l
+                return l
+        if len(self.stack) > 0:
+            element_name = self.stack[-1]
+            e = Element(self.connection, element_name, self.stack, self,
+                        self.list_marker, self.item_marker,
+                        self.pythonize_name)
+            self[self.get_name(element_name)] = e
+            return (element_name, e)
+        else:
+            return None
+
+    def endElement(self, name, value, connection):
+        if len(self.stack) > 0:
+            self.stack.pop()
+        value = value.strip()
+        if value:
+            if isinstance(self.parent, Element):
+                self.parent[self.get_name(name)] = value
+            elif isinstance(self.parent, ListElement):
+                self.parent.append(value)
+
+
+class ListElement(list):
+
+    def __init__(self, connection=None, element_name=None,
+                 list_marker=['Set'], item_marker=('member', 'item'),
+                 pythonize_name=False):
+        list.__init__(self)
+        self.connection = connection
+        self.element_name = element_name
+        self.list_marker = list_marker
+        self.item_marker = item_marker
+        self.pythonize_name = pythonize_name
+
+    def get_name(self, name):
+        if self.pythonize_name:
+            name = utils.pythonize_name(name)
+        return name
+
+    def startElement(self, name, attrs, connection):
+        for lm in self.list_marker:
+            if name.endswith(lm):
+                l = ListElement(self.connection, name,
+                                self.list_marker, self.item_marker,
+                                self.pythonize_name)
+                setattr(self, self.get_name(name), l)
+                return l
+        if name in self.item_marker:
+            e = Element(self.connection, name, parent=self,
+                        list_marker=self.list_marker,
+                        item_marker=self.item_marker,
+                        pythonize_name=self.pythonize_name)
+            self.append(e)
+            return e
+        else:
+            return None
+
+    def endElement(self, name, value, connection):
+        if name == self.element_name:
+            if len(self) > 0:
+                empty = []
+                for e in self:
+                    if isinstance(e, Element):
+                        if len(e) == 0:
+                            empty.append(e)
+                for e in empty:
+                    self.remove(e)
+        else:
+            setattr(self, self.get_name(name), value)

third_party/boto/core/service.py

+# Copyright (c) 2012 Mitch Garnaat http://garnaat.org/
+# Copyright (c) 2012 Amazon.com, Inc. or its affiliates.
+# All Rights Reserved
+#
+# Permission is hereby granted, free of charge, to any person obtaining a
+# copy of this software and associated documentation files (the
+# "Software"), to deal in the Software without restriction, including
+# without limitation the rights to use, copy, modify, merge, publish, dis-
+# tribute, sublicense, and/or sell copies of the Software, and to permit
+# persons to whom the Software is furnished to do so, subject to the fol-
+# lowing conditions:
+#
+# The above copyright notice and this permission notice shall be included
+# in all copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS
+# OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABIL-
+# ITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT
+# SHALL THE AUTHOR BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
+# WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+# IN THE SOFTWARE.
+#
+
+import requests
+from .auth import SigV2Auth
+from .credentials import get_credentials
+from .dictresponse import Element, XmlHandler
+
+
+class Service(object):
+    """
+    This is a simple example service that connects to the EC2 endpoint
+    and supports a single request (DescribeInstances) to show how to
+    use the requests-based code rather than the standard boto code which
+    is based on httplib.  At the moment, the only auth mechanism
+    supported is SigV2.
+    """
+
+    def __init__(self, host='https://ec2.us-east-1.amazonaws.com',
+                 path='/', api_version='2012-03-01', persona=None):
+        self.credentials = get_credentials(persona)
+        self.auth = SigV2Auth(self.credentials, api_version=api_version)
+        self.host = host
+        self.path = path
+
+    def get_response(self, params, list_marker=None):
+        r = requests.post(self.host, params=params,
+                          hooks={'args': self.auth.add_auth})
+        r.encoding = 'utf-8'
+        body = r.text.encode('utf-8')
+        e = Element(list_marker=list_marker, pythonize_name=True)
+        h = XmlHandler(e, self)
+        h.parse(body)
+        return e
+
+    def build_list_params(self, params, items, label):
+        if isinstance(items, str):
+            items = [items]
+        for i in range(1, len(items) + 1):
+            params['%s.%d' % (label, i)] = items[i - 1]
+
+    def describe_instances(self, instance_ids=None):
+        params = {}
+        if instance_ids:
+            self.build_list_params(params, instance_ids, 'InstanceId')
+        return self.get_response(params)

third_party/boto/datapipeline/exceptions.py

+# Copyright (c) 2012 Amazon.com, Inc. or its affiliates.  All Rights Reserved
+#
+# Permission is hereby granted, free of charge, to any person obtaining a
+# copy of this software and associated documentation files (the
+# "Software"), to deal in the Software without restriction, including
+# without limitation the rights to use, copy, modify, merge, publish, dis-
+# tribute, sublicense, and/or sell copies of the Software, and to permit
+# persons to whom the Software is furnished to do so, subject to the fol-
+# lowing conditions:
+#
+# The above copyright notice and this permission notice shall be included
+# in all copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS
+# OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABIL-
+# ITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT
+# SHALL THE AUTHOR BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
+# WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+# IN THE SOFTWARE.
+#
+from boto.exception import JSONResponseError
+
+
+class PipelineDeletedException(JSONResponseError):
+    pass
+
+
+class InvalidRequestException(JSONResponseError):
+    pass
+
+
+class TaskNotFoundException(JSONResponseError):
+    pass
+
+
+class PipelineNotFoundException(JSONResponseError):
+    pass
+
+
+class InternalServiceError(JSONResponseError):
+    pass

third_party/boto/ecs/__init__.py

+# Copyright (c) 2010 Chris Moyer http://coredumped.org/
+#
+# Permission is hereby granted, free of charge, to any person obtaining a
+# copy of this software and associated documentation files (the
+# "Software"), to deal in the Software without restriction, including
+# without limitation the rights to use, copy, modify, merge, publish, dis-
+# tribute, sublicense, and/or sell copies of the Software, and to permit
+# persons to whom the Software is furnished to do so, subject to the fol-
+# lowing conditions:
+#
+# The above copyright notice and this permission notice shall be included
+# in all copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS
+# OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABIL-
+# ITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT
+# SHALL THE AUTHOR BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, 
+# WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+# IN THE SOFTWARE.
+
+import boto
+from boto.connection import AWSQueryConnection, AWSAuthConnection
+import time
+import urllib
+import xml.sax
+from boto.ecs.item import ItemSet
+from boto import handler
+
+class ECSConnection(AWSQueryConnection):
+    """
+    ECommerce Connection
+
+    For more information on how to use this module see:
+
+    http://blog.coredumped.org/2010/09/search-for-books-on-amazon-using-boto.html
+    """
+
+    APIVersion = '2010-11-01'
+
+    def __init__(self, aws_access_key_id=None, aws_secret_access_key=None,
+                 is_secure=True, port=None, proxy=None, proxy_port=None,
+                 proxy_user=None, proxy_pass=None, host='ecs.amazonaws.com',
+                 debug=0, https_connection_factory=None, path='/'):
+        AWSQueryConnection.__init__(self, aws_access_key_id, aws_secret_access_key,
+                                    is_secure, port, proxy, proxy_port, proxy_user, proxy_pass,
+                                    host, debug, https_connection_factory, path)
+
+    def _required_auth_capability(self):
+        return ['ecs']
+
+    def get_response(self, action, params, page=0, itemSet=None):
+        """
+        Utility method to handle calls to ECS and parsing of responses.
+        """
+        params['Service'] = "AWSECommerceService"
+        params['Operation'] = action
+        if page:
+            params['ItemPage'] = page
+        response = self.make_request(None, params, "/onca/xml")
+        body = response.read()
+        boto.log.debug(body)
+
+        if response.status != 200:
+            boto.log.error('%s %s' % (response.status, response.reason))
+            boto.log.error('%s' % body)
+            raise self.ResponseError(response.status, response.reason, body)
+
+        if itemSet == None:
+            rs = ItemSet(self, action, params, page)
+        else:
+            rs = itemSet
+        h = handler.XmlHandler(rs, self)
+        xml.sax.parseString(body, h)
+        return rs
+
+    #
+    # Group methods
+    #
+    
+    def item_search(self, search_index, **params):
+        """
+        Returns items that satisfy the search criteria, including one or more search 
+        indices.
+
+        For a full list of search terms, 
+        :see: http://docs.amazonwebservices.com/AWSECommerceService/2010-09-01/DG/index.html?ItemSearch.html
+        """
+        params['SearchIndex'] = search_index
+        return self.get_response('ItemSearch', params)

third_party/boto/ecs/item.py

+# Copyright (c) 2010 Chris Moyer http://coredumped.org/
+#
+# Permission is hereby granted, free of charge, to any person obtaining a
+# copy of this software and associated documentation files (the
+# "Software"), to deal in the Software without restriction, including
+# without limitation the rights to use, copy, modify, merge, publish, dis-
+# tribute, sublicense, and/or sell copies of the Software, and to permit
+# persons to whom the Software is furnished to do so, subject to the fol-
+# lowing conditions:
+#
+# The above copyright notice and this permission notice shall be included
+# in all copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS
+# OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABIL-
+# ITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT
+# SHALL THE AUTHOR BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, 
+# WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+# IN THE SOFTWARE.
+
+
+import xml.sax
+import cgi
+from StringIO import StringIO
+
+class ResponseGroup(xml.sax.ContentHandler):
+    """A Generic "Response Group", which can
+    be anything from the entire list of Items to 
+    specific response elements within an item"""
+
+    def __init__(self, connection=None, nodename=None):
+        """Initialize this Item"""
+        self._connection = connection
+        self._nodename = nodename
+        self._nodepath = []
+        self._curobj = None
+        self._xml = StringIO()
+
+    def __repr__(self):
+        return '<%s: %s>' % (self.__class__.__name__, self.__dict__)
+
+    #
+    # Attribute Functions
+    #
+    def get(self, name):
+        return self.__dict__.get(name)
+    
+    def set(self, name, value):
+        self.__dict__[name] = value
+
+    def to_xml(self):
+        return "<%s>%s</%s>" % (self._nodename, self._xml.getvalue(), self._nodename)
+
+    #
+    # XML Parser functions
+    #
+    def startElement(self, name, attrs, connection):
+        self._xml.write("<%s>" % name)
+        self._nodepath.append(name)
+        if len(self._nodepath) == 1:
+            obj = ResponseGroup(self._connection)
+            self.set(name, obj)
+            self._curobj = obj
+        elif self._curobj:
+            self._curobj.startElement(name, attrs, connection)
+        return None
+
+    def endElement(self, name, value, connection):
+        self._xml.write("%s</%s>" % (cgi.escape(value).replace("&amp;amp;", "&amp;"), name))
+        if len(self._nodepath) == 0:
+            return
+        obj = None
+        curval = self.get(name)
+        if len(self._nodepath) == 1:
+            if value or not curval:
+                self.set(name, value)
+            if self._curobj:
+                self._curobj = None
+        #elif len(self._nodepath) == 2:
+            #self._curobj = None
+        elif self._curobj:
+            self._curobj.endElement(name, value, connection)
+        self._nodepath.pop()
+        return None
+
+
+class Item(ResponseGroup):
+    """A single Item"""
+
+    def __init__(self, connection=None):
+        """Initialize this Item"""
+        ResponseGroup.__init__(self, connection, "Item")
+
+class ItemSet(ResponseGroup):
+    """A special ResponseGroup that has built-in paging, and
+    only creates new Items on the "Item" tag"""
+
+    def __init__(self, connection, action, params, page=0):
+        ResponseGroup.__init__(self, connection, "Items")
+        self.objs = []
+        self.iter = None
+        self.page = page
+        self.action = action
+        self.params = params
+        self.curItem = None
+        self.total_results = 0
+        self.total_pages = 0
+
+    def startElement(self, name, attrs, connection):
+        if name == "Item":
+            self.curItem = Item(self._connection)
+        elif self.curItem != None:
+            self.curItem.startElement(name, attrs, connection)
+        return None
+
+    def endElement(self, name, value, connection):
+        if name == 'TotalResults':
+            self.total_results = value
+        elif name == 'TotalPages':
+            self.total_pages = value
+        elif name == "Item":
+            self.objs.append(self.curItem)
+            self._xml.write(self.curItem.to_xml())
+            self.curItem = None
+        elif self.curItem != None:
+            self.curItem.endElement(name, value, connection)
+        return None
+
+    def next(self):
+        """Special paging functionality"""
+        if self.iter == None:
+            self.iter = iter(self.objs)
+        try:
+            return self.iter.next()
+        except StopIteration:
+            self.iter = None
+            self.objs = []
+            if int(self.page) < int(self.total_pages):
+                self.page += 1
+                self._connection.get_response(self.action, self.params, self.page, self)
+                return self.next()
+            else:
+                raise
+
+    def __iter__(self):
+        return self
+
+    def to_xml(self):
+        """Override to first fetch everything"""
+        for item in self:
+            pass
+        return ResponseGroup.to_xml(self)

third_party/boto/roboto/__init__.py

+#