Metadata-Version: 2.1
Name: fs-gcsfs
Version: 1.4.0b1
Summary: A PyFilesystem interface to Google Cloud Storage
Home-page: https://github.com/Othoz/gcsfs
Author: Othoz GmbH
Author-email: wiesner@othoz.com
License: MIT
Project-URL: Bug Tracker, https://github.com/Othoz/gcsfs/issues
Project-URL: Documentation, http://fs-gcsfs.readthedocs.io/en/latest/
Description: 
        GCSFS
        =====
        
        A Python filesystem abstraction of Google Cloud Storage (GCS) implemented as a `PyFilesystem2 <https://github.com/PyFilesystem/pyfilesystem2>`__ extension.
        
        
        .. image:: https://img.shields.io/pypi/v/fs-gcsfs.svg
            :target: https://pypi.org/project/fs-gcsfs/
        
        .. image:: https://img.shields.io/pypi/pyversions/fs-gcsfs.svg
            :target: https://pypi.org/project/fs-gcsfs/
        
        .. image:: https://travis-ci.org/Othoz/gcsfs.svg?branch=master
            :target: https://travis-ci.org/Othoz/gcsfs
        
        .. image:: https://readthedocs.org/projects/fs-gcsfs/badge/?version=latest
            :target: https://fs-gcsfs.readthedocs.io/en/latest/?badge=latest
        
        .. image:: https://api.codacy.com/project/badge/Coverage/f863cb8afe7943b5ba8c818bbae2e6d2
            :target: https://www.codacy.com/app/Othoz/gcsfs?utm_source=github.com&amp;utm_medium=referral&amp;utm_content=Othoz/gcsfs&amp;utm_campaign=Badge_Coverage
        
        .. image:: https://api.codacy.com/project/badge/Grade/f863cb8afe7943b5ba8c818bbae2e6d2
            :target: https://www.codacy.com/app/Othoz/gcsfs?utm_source=github.com&amp;utm_medium=referral&amp;utm_content=Othoz/gcsfs&amp;utm_campaign=Badge_Grade
        
        
        With GCSFS, you can interact with `Google Cloud Storage <https://cloud.google.com/storage/>`__ as if it was a regular filesystem.
        
        Apart from the nicer interface, this will highly decouple your code from the underlying storage mechanism: Exchanging the storage backend with an
        `in-memory filesystem <https://pyfilesystem2.readthedocs.io/en/latest/reference/memoryfs.html>`__ for testing or any other
        filesystem like `S3FS <https://github.com/pyfilesystem/s3fs>`__ becomes as easy as replacing ``gs://bucket_name`` with ``mem://`` or ``s3://bucket_name``.
        
        For a full reference on all the PyFilesystem possibilities, take a look at the
        `PyFilesystem Docs <https://pyfilesystem2.readthedocs.io/en/latest/index.html>`__!
        
        
        Documentation
        -------------
        
        -  `GCSFS Documentation <http://fs-gcsfs.readthedocs.io/en/latest/>`__
        -  `PyFilesystem Wiki <https://www.pyfilesystem.org>`__
        -  `PyFilesystem Reference <https://docs.pyfilesystem.org/en/latest/reference/base.html>`__
        
        
        Installing
        ----------
        
        Install the latest GCSFS version by running::
        
            $ pip install fs-gcsfs
        
        Or in case you are using conda::
        
            $ conda install -c conda-forge fs-gcsfs
        
        
        Examples
        --------
        
        Instantiating a filesystem on Google Cloud Storage (for a full reference visit the
        `Documentation <http://fs-gcsfs.readthedocs.io/en/latest/index.html#reference>`__):
        
        .. code-block:: python
        
            from fs_gcsfs import GCSFS
            gcsfs = GCSFS(bucket_name="mybucket")
        
        
        Alternatively you can use a `FS URL <https://pyfilesystem2.readthedocs.io/en/latest/openers.html>`__ to open up a filesystem:
        
        .. code-block:: python
        
            from fs import open_fs
            gcsfs = open_fs("gs://mybucket/root_path?project=test&api_endpoint=http%3A//localhost%3A8888&strict=False")
        
        
        Supported query parameters are:
        
        - `project` (str): Google Cloud project to use
        - `api_endpoint` (str): URL-encoded endpoint that will be passed to the GCS client's `client_options <https://googleapis.dev/python/google-api-core/latest/client_options.html#google.api_core.client_options.ClientOptions>`__
        - `strict` ("True" or "False"): Whether GCSFS will be opened in strict mode
        
        
        You can use GCSFS like your local filesystem:
        
        .. code-block:: python
        
            >>> from fs_gcsfs import GCSFS
            >>> gcsfs = GCSFS(bucket_name="mybucket")
            >>> gcsfs.tree()
            ├── foo
            │   ├── bar
            │   │   ├── file1.txt
            │   │   └── file2.csv
            │   └── baz
            │       └── file3.txt
            └── file4.json
            >>> gcsfs.listdir("foo")
            ["bar", "baz"]
            >>> gcsfs.isdir("foo/bar")
            True
        
        
        Uploading a file is as easy as:
        
        .. code-block:: python
        
            from fs_gcsfs import GCSFS
            gcsfs = GCSFS(bucket_name="mybucket")
            with open("local/path/image.jpg", "rb") as local_file:
                with gcsfs.open("path/on/bucket/image.jpg", "wb") as gcs_file:
                    gcs_file.write(local_file.read())
        
        
        You can even sync an entire bucket on your local filesystem by using PyFilesystem's utility methods:
        
        .. code-block:: python
        
            from fs_gcsfs import GCSFS
            from fs.osfs import OSFS
            from fs.copy import copy_fs
        
            gcsfs = GCSFS(bucket_name="mybucket")
            local_fs = OSFS("local/path")
        
            copy_fs(gcsfs, local_fs)
        
        
        For exploring all the possibilities of GCSFS and other filesystems implementing the PyFilesystem interface, we recommend visiting the official
        `PyFilesystem Docs <https://pyfilesystem2.readthedocs.io/en/latest/index.html>`__!
        
        
        Development
        -----------
        
        To develop on this project make sure you have `pipenv <https://pipenv.readthedocs.io/en/latest/>`__ installed
        and run the following from the root directory of the project::
        
            $ pipenv install --dev --three
        
        This will create a virtualenv with all packages and dev-packages installed.
        
        
        Tests
        -----
        All CI tests run against an actual GCS bucket provided by `Othoz <http://othoz.com/>`__.
        
        In order to run the tests against your own bucket,
        make sure to set up a `Service Account <https://cloud.google.com/iam/docs/service-accounts>`__ with all necessary permissions:
        
        - storage.objects.get
        - storage.objects.list
        - storage.objects.create
        - storage.objects.update
        - storage.objects.delete
        
        All five permissions listed above are e.g. included in the `predefined Cloud Storage IAM Role <https://cloud.google.com/storage/docs/access-control/iam-roles>`__ ``roles/storage.objectAdmin``.
        
        Expose your bucket name as an environment variable ``$TEST_BUCKET`` and run the tests via::
        
            $ pipenv run pytest
        
        Note that the tests mostly wait for I/O, therefore it makes sense to highly parallelize them with `xdist <https://github.com/pytest-dev/pytest-xdist>`__, e.g. by running the tests with::
        
            $ pipenv run pytest -n 10
        
        
        Credits
        -------
        
        Credits go to `S3FS <https://github.com/PyFilesystem/s3fs>`__ which was the main source of inspiration and shares a lot of code with GCSFS.
Keywords: pyfilesystem,filesystem,google,gcs,google cloud storage
Platform: UNKNOWN
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.5
Classifier: Programming Language :: Python :: 3.6
Classifier: Programming Language :: Python :: 3.7
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: Implementation
Classifier: Topic :: System :: Filesystems
Requires-Python: >=3.5
Description-Content-Type: text/x-rst
