Schema validation just got Pythonic =============================================================================== **schema** is a library for validating Python data structures, such as those obtained from config-files, forms, external services or command-line parsing, converted from JSON/YAML (or something else) to Python data-types. .. image:: https://secure.travis-ci.org/keleshev/schema.png?branch=master :target: https://travis-ci.org/keleshev/schema .. image:: https://img.shields.io/codecov/c/github/keleshev/schema.svg :target: http://codecov.io/github/keleshev/schema Example ---------------------------------------------------------------------------- Here is a quick example to get a feeling of **schema**, validating a list of entries with personal information: .. code:: python >>> from schema import Schema, And, Use, Optional >>> schema = Schema([{'name': And(str, len), ... 'age': And(Use(int), lambda n: 18 <= n <= 99), ... Optional('sex'): And(str, Use(str.lower), ... lambda s: s in ('male', 'female'))}]) >>> data = [{'name': 'Sue', 'age': '28', 'sex': 'FEMALE'}, ... {'name': 'Sam', 'age': '42'}, ... {'name': 'Sacha', 'age': '20', 'sex': 'Male'}] >>> validated = schema.validate(data) >>> assert validated == [{'name': 'Sue', 'age': 28, 'sex': 'female'}, ... {'name': 'Sam', 'age': 42}, ... {'name': 'Sacha', 'age' : 20, 'sex': 'male'}] If data is valid, ``Schema.validate`` will return the validated data (optionally converted with `Use` calls, see below). If data is invalid, ``Schema`` will raise ``SchemaError`` exception. Installation ------------------------------------------------------------------------------- Use `pip <http://pip-installer.org>`_ or easy_install:: pip install schema Alternatively, you can just drop ``schema.py`` file into your project—it is self-contained. - **schema** is tested with Python 2.6, 2.7, 3.2, 3.3 and PyPy. - **schema** follows `semantic versioning <http://semver.org>`_. How ``Schema`` validates data ------------------------------------------------------------------------------- Types ~~~~~ If ``Schema(...)`` encounters a type (such as ``int``, ``str``, ``object``, etc.), it will check if the corresponding piece of data is an instance of that type, otherwise it will raise ``SchemaError``. .. code:: python >>> from schema import Schema >>> Schema(int).validate(123) 123 >>> Schema(int).validate('123') Traceback (most recent call last): ... SchemaUnexpectedTypeError: '123' should be instance of 'int' >>> Schema(object).validate('hai') 'hai' Callables ~~~~~~~~~ If ``Schema(...)`` encounters a callable (function, class, or object with ``__call__`` method) it will call it, and if its return value evaluates to ``True`` it will continue validating, else—it will raise ``SchemaError``. .. code:: python >>> import os >>> Schema(os.path.exists).validate('./') './' >>> Schema(os.path.exists).validate('./non-existent/') Traceback (most recent call last): ... SchemaError: exists('./non-existent/') should evaluate to True >>> Schema(lambda n: n > 0).validate(123) 123 >>> Schema(lambda n: n > 0).validate(-12) Traceback (most recent call last): ... SchemaError: <lambda>(-12) should evaluate to True "Validatables" ~~~~~~~~~~~~~~ If ``Schema(...)`` encounters an object with method ``validate`` it will run this method on corresponding data as ``data = obj.validate(data)``. This method may raise ``SchemaError`` exception, which will tell ``Schema`` that that piece of data is invalid, otherwise—it will continue validating. An example of "validatable" is ``Regex``, that tries to match a string or a buffer with the given regular expression (itself as a string, buffer or compiled regex ``SRE_Pattern``): .. code:: python >>> from schema import Regex >>> import re >>> Regex(r'^foo').validate('foobar') 'foobar' >>> Regex(r'^[A-Z]+$', flags=re.I).validate('those-dashes-dont-match') Traceback (most recent call last): ... SchemaError: Regex('^[A-Z]+$', flags=re.IGNORECASE) does not match 'those-dashes-dont-match' For a more general case, you can use ``Use`` for creating such objects. ``Use`` helps to use a function or type to convert a value while validating it: .. code:: python >>> from schema import Use >>> Schema(Use(int)).validate('123') 123 >>> Schema(Use(lambda f: open(f, 'a'))).validate('LICENSE-MIT') <open file 'LICENSE-MIT', mode 'a' at 0x...> Dropping the details, ``Use`` is basically: .. code:: python class Use(object): def __init__(self, callable_): self._callable = callable_ def validate(self, data): try: return self._callable(data) except Exception as e: raise SchemaError('%r raised %r' % (self._callable.__name__, e)) Now you can write your own validation-aware classes and data types. Lists, similar containers ~~~~~~~~~~~~~~~~~~~~~~~~~ If ``Schema(...)`` encounters an instance of ``list``, ``tuple``, ``set`` or ``frozenset``, it will validate contents of corresponding data container against schemas listed inside that container: .. code:: python >>> Schema([1, 0]).validate([1, 1, 0, 1]) [1, 1, 0, 1] >>> Schema((int, float)).validate((5, 7, 8, 'not int or float here')) Traceback (most recent call last): ... SchemaError: Or(<type 'int'>, <type 'float'>) did not validate 'not int or float here' 'not int or float here' should be instance of 'float' Dictionaries ~~~~~~~~~~~~ If ``Schema(...)`` encounters an instance of ``dict``, it will validate data key-value pairs: .. code:: python >>> d = Schema({'name': str, ... 'age': lambda n: 18 <= n <= 99}).validate({'name': 'Sue', 'age': 28}) >>> assert d == {'name': 'Sue', 'age': 28} You can specify keys as schemas too: .. code:: python >>> schema = Schema({str: int, # string keys should have integer values ... int: None}) # int keys should be always None >>> data = schema.validate({'key1': 1, 'key2': 2, ... 10: None, 20: None}) >>> schema.validate({'key1': 1, ... 10: 'not None here'}) Traceback (most recent call last): ... SchemaError: Key '10' error: None does not match 'not None here' This is useful if you want to check certain key-values, but don't care about other: .. code:: python >>> schema = Schema({'<id>': int, ... '<file>': Use(open), ... str: object}) # don't care about other str keys >>> data = schema.validate({'<id>': 10, ... '<file>': 'README.rst', ... '--verbose': True}) You can mark a key as optional as follows: .. code:: python >>> from schema import Optional >>> Schema({'name': str, ... Optional('occupation'): str}).validate({'name': 'Sam'}) {'name': 'Sam'} ``Optional`` keys can also carry a ``default``, to be used when no key in the data matches: .. code:: python >>> from schema import Optional >>> Schema({Optional('color', default='blue'): str, ... str: str}).validate({'texture': 'furry'} ... ) == {'color': 'blue', 'texture': 'furry'} True Defaults are used verbatim, not passed through any validators specified in the value. **schema** has classes ``And`` and ``Or`` that help validating several schemas for the same data: .. code:: python >>> from schema import And, Or >>> Schema({'age': And(int, lambda n: 0 < n < 99)}).validate({'age': 7}) {'age': 7} >>> Schema({'password': And(str, lambda s: len(s) > 6)}).validate({'password': 'hai'}) Traceback (most recent call last): ... SchemaError: Key 'password' error: <lambda>('hai') should evaluate to True >>> Schema(And(Or(int, float), lambda x: x > 0)).validate(3.1415) 3.1415 Extra Keys ~~~~~~~~~~ The ``Schema(...)`` parameter ``ignore_extra_keys`` causes validation to ignore extra keys in a dictionary, and also to not return them after validating. .. code:: python >>> schema = Schema({'name': str}, ignore_extra_keys=True) >>> schema.validate({'name': 'Sam', 'age': '42'}) {'name': 'Sam'} If you would like any extra keys returned, use ``object: object`` as one of the key/value pairs, which will match any key and any value. Otherwise, extra keys will raise a ``SchemaError``. User-friendly error reporting ------------------------------------------------------------------------------- You can pass a keyword argument ``error`` to any of validatable classes (such as ``Schema``, ``And``, ``Or``, ``Regex``, ``Use``) to report this error instead of a built-in one. .. code:: python >>> Schema(Use(int, error='Invalid year')).validate('XVII') Traceback (most recent call last): ... SchemaError: Invalid year You can see all errors that occurred by accessing exception's ``exc.autos`` for auto-generated error messages, and ``exc.errors`` for errors which had ``error`` text passed to them. You can exit with ``sys.exit(exc.code)`` if you want to show the messages to the user without traceback. ``error`` messages are given precedence in that case. A JSON API example ------------------------------------------------------------------------------- Here is a quick example: validation of `create a gist <http://developer.github.com/v3/gists/>`_ request from github API. .. code:: python >>> gist = '''{"description": "the description for this gist", ... "public": true, ... "files": { ... "file1.txt": {"content": "String file contents"}, ... "other.txt": {"content": "Another file contents"}}}''' >>> from schema import Schema, And, Use, Optional >>> import json >>> gist_schema = Schema(And(Use(json.loads), # first convert from JSON ... # use basestring since json returns unicode ... {Optional('description'): basestring, ... 'public': bool, ... 'files': {basestring: {'content': basestring}}})) >>> gist = gist_schema.validate(gist) # gist: {u'description': u'the description for this gist', u'files': {u'file1.txt': {u'content': u'String file contents'}, u'other.txt': {u'content': u'Another file contents'}}, u'public': True} Using **schema** with `docopt <http://github.com/docopt/docopt>`_ ------------------------------------------------------------------------------- Assume you are using **docopt** with the following usage-pattern: Usage: my_program.py [--count=N] <path> <files>... and you would like to validate that ``<files>`` are readable, and that ``<path>`` exists, and that ``--count`` is either integer from 0 to 5, or ``None``. Assuming **docopt** returns the following dict: .. code:: python >>> args = {'<files>': ['LICENSE-MIT', 'setup.py'], ... '<path>': '../', ... '--count': '3'} this is how you validate it using ``schema``: .. code:: python >>> from schema import Schema, And, Or, Use >>> import os >>> s = Schema({'<files>': [Use(open)], ... '<path>': os.path.exists, ... '--count': Or(None, And(Use(int), lambda n: 0 < n < 5))}) >>> args = s.validate(args) >>> args['<files>'] [<open file 'LICENSE-MIT', mode 'r' at 0x...>, <open file 'setup.py', mode 'r' at 0x...>] >>> args['<path>'] '../' >>> args['--count'] 3 As you can see, **schema** validated data successfully, opened files and converted ``'3'`` to ``int``.