arangopipe.arangopipe_storage package

Submodules

arangopipe.arangopipe_storage.arangopipe_admin_api module

Created on Thu Apr 25 09:30:33 2019

@author: Rajiv Sambasivan

class arangopipe.arangopipe_storage.arangopipe_admin_api.ArangoPipeAdmin(reuse_connection=True, config=None, persist_conn=True, client_url=None)

Bases: object

add_edge_definition_to_arangopipe(edge_col_name, edge_name, from_vertex_name, to_vertex_name)
add_edges_to_arangopipe(edge_col_name, from_vertex_list, to_vertex_list)
add_vertex_to_arangopipe(vertex_to_create)
check_repeated_creation(api_data)
create_config()
create_db(db_srv_host, db_srv_port, db_serv_name, db_end_point, db_dbName, db_user_name, db_password, db_conn_protocol)
create_enterprise_ml_graph(db_replication_factor)
delete_all_databases(preserve=['arangopipe', 'facebook_db', 'fb_node2vec_db', 'node2vecdb', '_system'])
delete_arangomldb()
delete_database(db_to_delete)
get_config()
has_edge(edge_name)
has_vertex(vertex_name)
register_deployment(dep_tag)
register_project(p)
remove_edge_definition_from_arangopipe(edge_name, purge=True)
remove_vertex_from_arangopipe(vertex_to_remove, purge=True)
set_connection_params(config)

arangopipe.arangopipe_storage.arangopipe_api module

Created on Sat Apr 13 08:35:58 2019

@author: Rajiv Sambasivan

class arangopipe.arangopipe_storage.arangopipe_api.ArangoPipe(config)

Bases: object

An instance of ArangoPipe is meant to be used to log a run of a pipeline execution. To use it: (1) Create a ArangoPipe object (2) Register your dataset with ArangoPipe (3) Register your featureset with ArangoPipe (4) Register you model with ArangoPipe

find_entity(attrib_name, attrib_value, asset_type)
get_collection_from_id(id_str)
get_config()
heart_beat()
init_graph()

Initialize a graph when an instance of ArangoPipe is provisioned.

insert_into_edge_type(edge_name, from_vdoc, to_vdoc, document=None)
insert_into_vertex_type(vertex_type_name, document)
is_valid_id_str(id_str)
log_run(ri)

Log a run. Logging a run requires specifying a dataset, featureset and a model against which this run is recored. A run records model parameters and model performance. The run object is probably most useful for the analysis of model performance with respect to a featureset, model hyper-parameters and a dataset.

log_serving_perf(sp, dep_tag, userid='authorized user')

Log serving performance against a deployed model. The user making the request needs to be authorized to log this performance update. A serving performance vertex is created and is linked with its deployment vertex

lookup_dataset(dataset_name)

Return a dataset identifier given a name. This can be used to get the dataset id that is used to log run information associated with execution of the pipeline.

lookup_entity(asset_name, asset_type)
lookup_entity_by_id(entity_id)
lookup_featureset(feature_set_name)

Return a featureset identifier given a name. This can be used to get the featureset id that is used to log run information associated with execution of the pipeline.

lookup_model(model_name)

Return a model identifier given a name. This can be used to get the model id that is used to log run information associated with execution of the pipeline.

lookup_modelparams(tag_value)

Return a model parameter result given a tag.

lookup_modelperf(tag_value)

Return a model dev performance given a tag.

register_dataset(ds_info, user_id='authorized_user')

Register a dataset. The operation requires specifying a user id. If the user id is permitted to register a dataset, then the registration proceeds, otherwise an unauthorized operation is indicated.

register_featureset(fs_info, dataset_id, user_id='authorized_user')

Register a featureset. ManagedServiceConnParamThe operation requires specifying a user id. If the user id is permitted to register a featureset, then the registration proceeds, otherwise an unauthorized operation is indicated.

register_model(mi, user_id='authorized_user', project='Wine-Quality-Regression-Modelling')

Register a model. The operation requires specifying a user id. If the user id is permitted to register a model, then the registration proceeds, otherwise an unauthorized operation is indicated.

arangopipe.arangopipe_storage.arangopipe_config module

Created on Fri Jun 14 09:17:50 2019

@author: Rajiv Sambasivan

class arangopipe.arangopipe_storage.arangopipe_config.ArangoPipeConfig

Bases: object

create_config(file_path)
create_connection_config(conn_params)
dump_data()
export_cfg(file_path)
get_cfg()
read_data()
set_cfg(new_cfg)

arangopipe.arangopipe_storage.custom_http_client module

Spyder Editor @author: Rajiv Sambasivan

class arangopipe.arangopipe_storage.custom_http_client.CustomHTTPClient(username, password)

Bases: arango.http.HTTPClient

My custom HTTP client with cool features.

create_session(host)

Return a new requests session given the host URL.

This method must be overridden by the user.

Parameters

host (str | unicode) – ArangoDB host URL.

Returns

Requests session object.

Return type

requests.Session

send_request(session, method, url, params=None, data=None, headers=None, auth=None)

Send an HTTP request.

This method must be overridden by the user.

Parameters

  • session (requests.Session) – Requests session object.

  • method (str | unicode) – HTTP method in lowercase (e.g. “post”).

  • url (str | unicode) – Request URL.

  • headers (dict) – Request headers.

  • params (dict) – URL (query) parameters.

  • data (str | unicode | bool | int | list | dict) – Request payload.

  • auth (tuple) – Username and password.

Returns

HTTP response.

Return type

arango.response.Response

arangopipe.arangopipe_storage.managed_service_conn_parameters module

Created on Thu Dec 5 09:05:19 2019

@author: Rajiv Sambasivan

class arangopipe.arangopipe_storage.managed_service_conn_parameters.ManagedServiceConnParam

Bases: object

property DB_CONN_PROTOCOL
property DB_NAME
property DB_NOTIFICATION_EMAIL
property DB_PASSWORD
property DB_REPLICATION_FACTOR
property DB_ROOT_USER
property DB_ROOT_USER_PASSWORD
property DB_SERVICE_END_POINT
property DB_SERVICE_HOST
property DB_SERVICE_NAME
property DB_SERVICE_PORT
property DB_USER_NAME

Module contents