fdi.pal package

Submodules

fdi.pal.backup module

fdi.pal.comparable module

class fdi.pal.comparable.Comparable(**kwds)[source]

Bases: object

compareTo(o)[source]

fdi.pal.context module

class fdi.pal.context.AbstractContext(*args, **kwds)[source]

Bases: object

A special kind of Product that can hold references to other Products.

This abstract product introduces the lazy loading and saving of references to Products or ProductRefs that it is holding. It remembers its state. http://herschel.esac.esa.int/hcss-doc-15.0/load/hcss_drm/api/herschel/ia/pal/Context.html

Sets rule to None if zInfo does not have refs, else to zInfo['refs'].

__init__(*args, **kwds)[source]

Sets rule to None if zInfo does not have refs, else to zInfo['refs'].

addRule(rule)[source]

Add to the rule that controls the products to be added into the context. The new rule will be old rule AND added rule.

applyRule(*args)[source]

returns True if the input ProductRef passes rule.

If refs is not specified in zInfo return True. If ``zInfo[‘refs’] is empty, return True.

Subclasses can override this method according to what is written in zInfo.

getAllRefs(recursive, includeContexts)[source]

Provides a set of the unique references stored in this context.

getRefs()[source]

Returns the reference container mapping

getRule()[source]

Get the rule that controls the products to be added into the context.

hasDirtyReferences(storage)[source]

Returns a logical to specify whether this context has dirty references or not.

static isContext(cls)[source]

Yields true if specified class belongs to the family of contexts.

isValid()[source]

Provides a mechanism to ensure whether it is valid to store this context in its current state.

readDataset(storage, table, defaultPoolId)[source]

Reads a dataset with information within this context that is normally not accessible from the normal Product interface.

property refs

Property

refsChanged()[source]

Indicates that the references have been changed in memory, which marks this context as dirty.

set(name, refs)[source]

add owner to RefContainer or add named references.

Name:

name of the new RefContainer. if is refs, set owner of refs to self.

setRefs(refs)[source]

Changes/Adds the mapping container that holds references.

setRule(rule)[source]

Set the rule that controls the products to be added into the context.

writeDataset(storage)[source]

Creates a dataset with information within this context that is normally not accessible from the normal Product interface.

class fdi.pal.context.Context(*args, **kwds)[source]

Bases: AbstractContext, BaseProduct

See docstring of AbstractContext.

__init__(*args, **kwds)[source]
applyRule(*args)[source]

returns True if the input ProductRef passes rule.

Default behavior is return the superclass.applyRule() result.

Subclasses can override this method according to what is written in zInfo.

getAllRefs(recursive=False, includeContexts=True, seen=None)[source]

Provides a set of the unique references stored in this Context.

This includes references that are contexts, but not the contents of these subcontexts. This is equivalent to getAllRefs(recursive=false, includeContexts= true). recursive - if true, include references in subcontexts includeContexts - if true, include references to contexts, not including this one

hasDirtyReferences(storage)[source]

Returns a logical to specify whether this context has dirty references or not.

exception fdi.pal.context.ContextRuleException[source]

Bases: ValueError

class fdi.pal.context.MapContext(*args, **kwds)[source]

Bases: Context

Allows grouping Products into a map of (String, ProductRef) pairs. New entries can be added if they comply to the adding rules of this context. The default behaviour is to allow adding any (String,ProductRef) given that rule is not set.

An example:

image     = ImageProduct(description="hi")
spectrum  = SpectrumProduct(description="there")
simple    = Product(description="everyone")

context=MapContext()
context.refs.put("x",ProductRef(image))
context.refs.put("y",ProductRef(spectrum))
context.refs.put("z",ProductRef(simple))
print context.refs.size() # 3
print context.refs.get('x').product.description # hi
print context.refs.get('y').product.description # there
print context.refs.get('z').product.description # everyone

It is possible to insert a ProductRef at a specific key in the MapContext. The same insertion behaviour is followed as for a Python dict, in that if there is already an existing ProductRef for the given key, that ProductRef is replaced with the new one:

product4=SpectrumProduct(description="everybody")
context.refs.put("y", ProductRef(product4))
product5=SpectrumProduct(description="here")
context.refs.put("a", ProductRef(product5))

print context.refs.get('x').product.description # hi
print context.refs.get('y').product.description # everybody
print context.refs.get('z').product.description # everyone
print context.refs.get('a').product.description # here

Note that the rules are only applied when putting an entry to the map!

Be aware that

  1. the put() method of the map view may throw a ContextRuleException if the data added to the context violates the rules applied to the context.

  2. the put() method of the map view may throw a ValueError if either of the arguments to the put() method are null.

    BaseProduct–Product

AbstractContext——Contex———MapContext

__init__(*args, **kwds)[source]
addRule(rule)[source]

Add to the rule that controls the products to be added into the context. The new rule will be old rule AND added rule.

applyRule(key, ref)[source]

returns True if the input key name and ProductRef pass rule.

Default behavior is return True if the fully qualified class name of the product which ref refers to equals zInfo['refs'][key], OR the superclass.applyRule() result.

Subclasses can override this method according to what is written in zInfo.

class fdi.pal.context.RefContainer(**kwds)[source]

Bases: ODict

A map where Rules of its owner Context are applied when put(k,v) is called, and the owner MapContext’s ID can be put to v’s parents list.

Implemented as an ODict that RefContainer has a _STID when json.loads’ed. A MapContext has a _data, which has a refs:RefContainer, which has a datap, which has a name:ProductRef. when used as context.refs.get(‘x’).product.description, the RefContainer is called with get() or __getitem__(), which calls superclass composite’s __getitem__()

__init__(**kwds)[source]
clear()[source]

remove all productRefs

get(key, *args, **kwds)[source]
getOwner()[source]

Returns the reference container mapping

property owner

Property

put(key, ref)

set label-ref pair after validating then add parent to the ref

set(key, ref)[source]

set label-ref pair after validating then add parent to the ref

setOwner(owner)[source]

records who owns this container.

Due to reason described in __setitem__ doc, existing refs will be set again upon owner set

size()[source]
string(level=0, tablefmt='grid', **kwds)
level:

default=0,

keyval:

default=`None`. If set to a string, ODict class name is not shown and a label of given string is shown with each key as ‘label key:

‘.
tablefmt:

=’rst’, tablefmt1=’simple’, tablefmt2=’simple’, matprint=None, trans=True, heavy=True

toString(level=0, tablefmt='grid', **kwds)[source]
level:

default=0,

keyval:

default=`None`. If set to a string, ODict class name is not shown and a label of given string is shown with each key as ‘label key:

‘.
tablefmt:

=’rst’, tablefmt1=’simple’, tablefmt2=’simple’, matprint=None, trans=True, heavy=True

txt(level=0, tablefmt='grid', **kwds)
level:

default=0,

keyval:

default=`None`. If set to a string, ODict class name is not shown and a label of given string is shown with each key as ‘label key:

‘.
tablefmt:

=’rst’, tablefmt1=’simple’, tablefmt2=’simple’, matprint=None, trans=True, heavy=True

fdi.pal.definable module

class fdi.pal.definable.Definable(**kwds)[source]

Bases: object

for items being able to be defined with a Definition.

getDefinition()[source]

Returns the definition associated to this definable item. mh: adopting http://herschel.esac.esa.int/hcss-doc-15.0/load/hcss_drm/api/index.html?herschel/ia/pal/ProductRef.html but parameterize definition does not seem worth it given the few numbers of implemented definitions

fdi.pal.dicthk module

class fdi.pal.dicthk.DictHk(**kwds)[source]

Bases: Taggable

Definition of services provided by a product storage supporting versioning.

getAllUrns()[source]

Returns a list of all URNs in the pool.

getTagUrnMap()[source]

Get the full tag->urn mappings.

mh: returns an iterator csdb: csdb/v1/storage/tag?tag=tag1,tag2

getTags(urn=None, datatype=None, sn=None)[source]

Get all of the tags that map to a given URN or a pair of data type and serial number.

Get all known tags if input arenot specified. mh: returns an iterator.

If datatype and sn are given, use them and ignore urn.

getUrn(tag)[source]

Gets the URNs corresponding to the given tag.

Returns an empty list if tag is not None and does not exist. curl -X GET “http://123.56.102.90:31702/csdb/v1/storage/info?urns=urn%3Apoolbs%3A20211018%3A1” -H “accept: /

getUrnObject(tag)[source]

Gets the URNobjects corresponding to the given tag. Returns an empty list if tag does not exist.

get_missing(urn, datatype, sn, no_check=False)

make URN(s) if datatype and sn(s) are given and vice versa.

Parameters:

no_check (bool) – Do not Check if datatype and sn are in the pool’s HK. Default is False

Returns:

  • tuple

  • str, str, int – Refer tp parseUrn.

Raises:
  • ValueError if urn not found or not from this pool.

  • KeyError if datatype does not exist.

  • IndexError if sn does not exist.

removeTag(tag)[source]

Remove the given tag from the tag and urn maps. # TODO in CSDB

removeUrn(urn=None, datatype=None, sn=None)[source]

Remove the given urn (or a pair of data type and serial number) from the tag and urn maps.

Only changes maps in memory, not to write on disk here.

removekey(key, thecontainer, thename, cross_ref_map, othername)[source]

Remove the given key from the map and the counterpart key in the correponding cross_referencing map.

setTag(tag, urn=None, datatype=None, sn=None)[source]

Sets the specified tag to the given URN or a pair of data type and serial number.

# TODO in CSDB

tagExists(tag)[source]

Tests if a tag exists.

fdi.pal.dicthk.add_tag_datatype_sn(tag, datatype, sn, dTypes=None, dTags=None)[source]

Static function to add a tag to datatype-sn to pool fmt 2.0

Parameters:
  • tag (str) – A tag. Multiple tags have to make multiple calls. None and empty tags are ignored.

  • datatype (str) – The class name of the data item, new or existing.

  • sn (int) – The serial number in integer.

  • dTypes (dict) – the first nested mapping of pool fmt 2.

  • dTags (dict) – the tag mapping of pool fmt 2.

fdi.pal.dicthk.get_missing(self, urn, datatype, sn, no_check=False)[source]

make URN(s) if datatype and sn(s) are given and vice versa.

Parameters:

no_check (bool) – Do not Check if datatype and sn are in the pool’s HK. Default is False

Returns:

  • tuple

  • str, str, int – Refer tp parseUrn.

Raises:
  • ValueError if urn not found or not from this pool.

  • KeyError if datatype does not exist.

  • IndexError if sn does not exist.

fdi.pal.httpclientpool module

fdi.pal.httppool module

class fdi.pal.httppool.HttpPool(**kwds)[source]

Bases: LocalPool

the pool will save all products locally on the host Http server.

creates file structure if there isn’t one. if there is, read and populate house-keeping records. create persistent files if not exist.

__init__(**kwds)[source]

creates file structure if there isn’t one. if there is, read and populate house-keeping records. create persistent files if not exist.

fdi.pal.localpool module

class fdi.pal.localpool.LocalPool(makenew=True, **kwds)[source]

Bases: ManagedPool

the pool will save all products in local computer.

Makenew:

when the pool does not exist, make a new one (``True`; default) or throws PoolNotFoundError (`False`).

creates file structure if there isn’t one. if there is, read and populate house-keeping records. create persistent files if not exist.

__init__(makenew=True, **kwds)[source]

creates file structure if there isn’t one. if there is, read and populate house-keeping records. create persistent files if not exist.

backup()[source]

make a tarfile string into a string

doLoad(resourcetype, index, start=None, end=None, serialize_out=False)[source]

does the action of loading.

serialize_out: if True returns contents in serialized form.

doRemove(resourcetype, index, **kwds)[source]

does the action of removal of product from pool.

doSave(resourcetype, index, data, tags=None, serialize_in=True, **kwds)[source]

does the media-specific saving.

index: int

doWipe()[source]

does the action of remove-all

getCacheInfo()[source]
getHead(ref)[source]

Returns the latest version of a given product, belonging to the first pool where the same track id is found.

getMetaByUrn(urn, resourcetype=None, index=None)[source]

Get all of the meta data belonging to a product of a given URN or a pair of data type and serial number.

mh: returns an iterator.

getMetaByUrnJson(urn, resourcetype, index)[source]

like getMetaByUrn but returns un-deserialized form.

readHK(hktype=None, serialize_out=False, all_versions=True)[source]

loads and returns the housekeeping data, or empty dict if not found.

hktype: one of the mappings listed in dicthk.HKDBS. serialize_out: if True return serialized form. Default is false.

readmmap(filename, start=None, end=None, close=False, check_time=False)[source]
restore(tar)[source]

untar the input file to this pool and return the file list.

setMetaByUrn(start, end, urn=None, datatype=None, sn=None)[source]

Sets the location of the meta data of the specified data to the given URN or a pair of data type and serial number.

Data:

usually serialized Product.

setup()[source]

Sets up LocalPool interals.

Make sure that self._poolname and self._poolurl are present.

writeHK(fp0=None, all_versions=True)[source]

save the housekeeping data to disk

writeJsonmmap(fp, data, serialize_in=True, serialize_out=False, close=False, check_time=False, meta_location=False, **kwds)[source]

write data in JSON from mmap file at fp.

register the file. Leave file open by default close. data: to be serialized and saved. serialize_out: if True returns contents in serialized form. :check_time: to check if file has not been written since we did last time. Default False. :meta_location: return the start and end offsets of metadata in data JSON. Default False. :return: int bytes written. If meta_location is `True`, adding int int start and end point offsets of metadata in seriaized data.

fdi.pal.localpool.wipeLocal(path)[source]

does the scheme-specific remove-all.

A new directory at path will be created.

fdi.pal.mempool module

class fdi.pal.mempool.MemPool(**kwds)[source]

Bases: ManagedPool

the pool will save all products in memory.

creates data structure if there isn’t one. if there is, read and populate house-keeping records. create persistent files if not exist.

__init__(**kwds)[source]

creates data structure if there isn’t one. if there is, read and populate house-keeping records. create persistent files if not exist.

doLoad(resourcetype, index, start=0, end=0, serialize_out=False)[source]

does the action of loadProduct. note that the index is given as a string.

doRemove(resourcetype, index, asyn=False)[source]

does the action of removal.

doSave(resourcetype, index, data, tags=None, serialize_in=True, **kwds)[source]

does the media-specific saving

doWipe()[source]

does the action of remove-all

getHead(ref)[source]

Returns the latest version of a given product, belonging to the first pool where the same track id is found.

getMetaByUrn(urn, resourcetype=None, index=None)[source]

Get all of the meta data belonging to a product of a given URN.

getPoolSpace()[source]

returns the map of this memory pool.

readHK(hktype=None, serialize_in=True, serialize_out=False)[source]

loads and returns the housekeeping data

hktype: one of ‘classes’, ‘tags’, ‘urns’ to return. default is None to return alldirs serialize_out: if True return serialized form. Default is false.

setMetaByUrn(data, urn, resourcetype=None, index=None)[source]

Sets the location of the meta data of the specified data to the given URN.

Data:

usually unserialized Product.

setup()[source]

Sets up MemPool interals.

make sure that self._poolname and self._poolurl are present.

writeHK()[source]

save the housekeeping data to mempool

fdi.pal.poolmanager module

class fdi.pal.poolmanager.PoolManager[source]

Bases: object

This class provides the means to reference ProductPool objects without having to hard-code the type of pool. For example, it could be desired to easily switch from one pool type to another.

This is done by calling the getPool() method, which will return an existing pool or create a new one if necessary.

PlacePaths = {'csdb': 'v1', 'file': '/tmp/httppool', 'http': '111.111.111.111:2222/fdi/v0.15', 'https': '111.111.111.111:2222/fdi/v0.15', 'mem': '/', 'server': '/tmp/httppool/data'}
classmethod getMap()[source]

Returns a poolname - poolobject map.

classmethod getPool(poolname=None, poolurl=None, pool=None, makenew=True, auth=None, client=None, **kwds)[source]

returns an instance of pool according to name or path of the pool.

Returns the pool object if the pool is registered. Creates the pool if it does not already exist. the same poolname-path always get the same pool. Http pools will be registered on the server side.

Pools registered are kept as long as the last reference remains. When the last is gone the pool gets :meth;`removed` d.

poolnamestr

name of the pool.

poolurlstr

If given the poolpath, scheme, place will be derived from it. if not given for making a new pool (i.e. when poolname is not a registered pool name. If poolname is missing it is derived from poolurl; if poolurl is also absent, ValueError will be raised.

pool: ProductPool

If auth and client are given they will substitute those of pool. If pool is not given, those will need to be given.

makenewbool

When the pool does not exist, make a new one (``True`; default) or throws PoolNotFoundError (`False`).

authstr

For remoteRegister.

clientdefault is None.

For remoteRegister.

kwdsdict

Passed to pool instanciation arg-list.

ProductPool:

The pool object.

classmethod getPoolurlMap()[source]

Gives the default poolurls of PoolManager.

classmethod isLoaded(poolname)[source]

Whether an item with the given id has been loaded (cached).

Returns:

the number of remaining week references if the pool is loaded. Returns 0 if poolname is not found in _GlobalPoolList or weakref count is 0.

items()[source]

Returns map’s items

classmethod remove(poolname, ignore_error=False)[source]

Remove from list and unregister remote pools.

returns 0 for successful removal, 1 for poolname not registered or referenced, still attempted to remove. > 1 for the number of weakrefs the pool still have, and removing failed.

classmethod removeAll(ignore_error=False)[source]

deletes all pools from the pool list, pools not wiped

classmethod save(poolname, poolobj)[source]
classmethod setPoolurlMap(new)[source]

Sets the default poolurls of PoolManager.

classmethod size()[source]

Gives the number of entries in this manager.

fdi.pal.poolmanager.remoteRegister(pool)[source]

if registered a pool’s auth and client will be used.

Note that a new http/csdb pool gets remoteRegistered before locally registered.

Parameter

poolHttpClientPool, PublicClientPool

Pool object to be registered on remote server and have client/session set.

authobject

Authorization object for the client. If given will substitute that of pool, if pool has auth.

clientflask.requests (testing), or requests.Session

The client. If given will substitute that of pool, if pool is given

fdi.pal.poolmanager.remoteUnregister(poolurl, auth=None, client=None)[source]

this method does not reference pool object.

fdi.pal.productpool module

class fdi.pal.productpool.ManagedPool(**kwds)[source]

Bases: ProductPool, DictHk

A ProductPool that manages its internal house keeping.

Creates and initializes a productpool.

  • poolname: if provided will override that in poolurl.

  • poolurl: needed to initialize.

dereference(ref)[source]

Decrement the reference count of a ProductRef.

doLoad(resourcetype, index, start=None, end=None, serialize_out=False)[source]

to be implemented by subclasses to do the action of loading

doRemove(resourcetype, index, ayn=False)[source]

to be implemented by subclasses to do the action of reemoving

doSave(resourcetype, index, data, tags=None, serialize_in=True, **kwds)[source]

to be implemented by subclasses to do the action of saving

doSelect(query, previous=None)[source]

to be implemented by subclasses to do the action of querying.

doWipe()[source]

to be implemented by subclasses to do the action of wiping.

exists(urn, resourcetype=None, index=None)[source]

Determines the existence of a product with specified URN.

getCacheInfo()[source]
getCount(typename=None)[source]

Return the number of URNs for the product type.

getMetaByUrn(urn=None, resourcetype=None, index=None)[source]

Get all of the meta data belonging to a product of a given URN.

mh: returns an iterator.

getPoolpath()[source]

Gets the poolpath of this pool.

poolpath is usually derived from poolurl received from PoolManager during initialization.

getProductClasses()[source]

Returns all Product classes found in this pool. mh: returns an iterator.

getReferenceCount(ref)[source]

Returns the reference count of a ProductRef.

isEmpty()[source]

Determines if the pool is empty.

loadDescriptors(urn, resourcetype=None, index=None)[source]

Loads the descriptors belonging to specified URN.

lockpath(op='w')[source]

Make lock path using transformed poolname as name.

meta(*args, **kwds)[source]

Loads the meta-data info belonging to the product of specified URN.

meta_filter(q, typename=None, reflist=None, urnlist=None, snlist=None, datatypes=None)[source]

returns filtered collection using the query.

q is a MetaQuery valid inputs: typename and ns list; productref list; urn list; datatypes dict.

Typename:

data type (class name)

Reflist:

list of ProductRefs

Urnlist:

list of URNs

Datatypes:

dict of {typename:sn_list}

prod_filter(q, cls=None, reflist=None, urnlist=None, snlist=None, datatypes=None)[source]

returns filtered collection using the query.

q: an AbstractQuery. valid inputs: cls and ns list; productref list; urn list; datatypes dict.

Cls:

type. data type

Reflist:

list of ProductRefs

Urnlist:

list of URNs

Datatypes:

dict of {cls:sn_list}

readHK()[source]

Subclass should overide this.

Returns:

Of 5 dicts that are the legacy self._classes, self._tags,
self._urns, and

self._dTypes, self._dTags

Return type:

tuple

reference(ref)[source]

Increment the reference count of a ProductRef.

saveOne(prd, tag, geturnobjs, serialize_in, serialize_out, res, **kwds)[source]

Save one product.

# get the latest HK

the new classes and tags.
dTypes combines old classes and urns, is defined as::
$class_name0:

‘currentSN’: $sn ‘sn’:

$sn0:

tags: [$tag1, $tag2, … meta: [$start, $end] refcnt: $count

$sn1:

tags: [$tag1, $tag2, … meta: [$start, $end] refcnt: $count

example::
‘foo.bar.Bar’:

‘currentSN’: 1 0:

‘tags’: [‘cat’, ‘white’] ‘meta’: [123, 456] ‘refcnt’: 0

1:

‘tags’: [‘dog’, ‘white’] ‘meta’: [321, 765] ‘refcnt’: 0

‘foo.baz.Baz’:

‘currentSN’: 34 34:

‘tags’: [‘tree’, ‘green’] ‘meta’: [100, 654] ‘refcnt’: 1

dTags differs from tag 1. uses dType:[sn], instead of urn, so there is no poolname anywhere, 2. simplify by removing second level dict::
$tag_name0:

$class_name1:[$sn1, $sn2…] $class_name2:[$sn3, …]

example::

‘cat’: { ‘foo.bar.Bar’:[0] } ‘white’: { ‘foo.bar.Bar’; [0, 1] } ‘dog’: …

res:

list of results.

serialize_out:

if True returns contents in serialized form.

list of the following: ProductRef. Urn if geturnobjs is set. if`serialze_out` is set for ProductRef no product metadata is stored in the returned instance. The result is also stored in the re parameter.

schematicLoad(resourcetype, index, start=None, end=None, serialize_out=False)[source]

do the scheme-specific loading

schematicRemove(urn=None, resourcetype=None, index=None, asyn=False, **kwds)[source]

do the scheme-specific removing

schematicSave(products, tag=None, geturnobjs=False, serialize_in=True, serialize_out=False, asyn=False, **kwds)[source]

do the scheme-specific saving.

Parameters:
  • product (BaseProduct, list) – Product or a list of them or ‘[ size1, prd, size2, prd2, …]’.

  • tag (str, list) – If given a tag, all products will be having this tag.

  • If a list tags are given to every one product then the

  • number of tags must not be the same to that of `product`. If

  • they are equal, each tag is goven to the product at the same

  • index in the `product` list.

  • serialize_out (bool) – if True returns contents in serialized form.

  • serialize_in (bool) – If set, product input is serialized.

Returns:

  • ProductRef (Product reference.)

  • Urn (If geturnobjs is set.)

  • str (If serialze_out is set, serialized form of ProductRef or URN.)

  • list (list of the above of input is a list.)

schematicSelect(query, previous=None)[source]

do the scheme-specific querying.

schematicWipe(asyn=False)[source]

do the scheme-specific wiping

setMetaByUrn(start, end, urn=None, datatype=None, sn=None)[source]

Sets the location of the meta data of the specified data to the given URN or a pair of data type and serial number.

Data:

usually un/serialized Product.

Return :urn: stirng :datatype: class name :sn: serial number in string.

setup()[source]
Sets up interal machiney of this Pool,

but only if self._poolname and self._poolurl are present, and other pre-requisits are met.

Subclasses should implement own setup(), and make sure that self._poolname and self._poolurl are present with ``

if <pre-requisit not met>: return True if super().setup(): return True

# super().setup() has done its things by now. <do setup> return False

``

returns: True if not both self._poolname and self._poolurl are present.

transformpath(path)[source]

override this to changes the output from the input one (default) to something else.

where(qw, prod='BaseProduct', urns=None)[source]
exception fdi.pal.productpool.PoolNotFoundError[source]

Bases: Exception

class fdi.pal.productpool.ProductPool(poolname='', poolurl='', **kwds)[source]

Bases: Definable, Taggable, Versionable

A mechanism that can store and retrieve Products.

A product pool should not be used directly by users. The general user should access data in a ProductPool through a ProductStorage instance.

When implementing a ProductPool, the following rules need to be applied:

  1. Pools must guarantee that a Product saved via the pool saveProduct(Product) method is stored persistently, and that method returns a unique identifier (URN). If it is not possible to save a Product, an IOException shall be raised.

  2. A saved Product can be retrieved using the loadProduct(Urn) method, using as the argument the same URN that assigned to that Product in the earlier saveProduct(Product) call. No other Product shall be retrievable by that same URN. If this is not possible, an IOException or GeneralSecurityException is raised.

  3. Pools should not implement functionality currently implemented in the core package. Specifically, it should not address functionality provided in the Context abstract class, and it should not implement versioning/cloning support.

Creates and initializes a productpool.

  • poolname: if provided will override that in poolurl.

  • poolurl: needed to initialize.

exception ParametersIncommpleteError[source]

Bases: Exception

__init__(poolname='', poolurl='', **kwds)[source]

Creates and initializes a productpool.

  • poolname: if provided will override that in poolurl.

  • poolurl: needed to initialize.

accept(visitor)[source]

Hook for adding functionality to object through visitor pattern.

property count

for property getter

dereference(ref)[source]

Decrement the reference count of a ProductRef.

XXX TODO

exists(urn)[source]

Determines the existence of a product with specified URN.

getCount(typename=None)[source]

Return the number of URNs for the product type.

getDefinition()[source]

Returns pool definition info which contains pool type and other pool specific configuration parameters

getId()[source]

Gets the identifier of this pool.

getPlace()[source]

Gets the place of this pool.

getPoolManager()[source]
getPoolname()[source]

Gets the poolname of this pool as an Object.

getPoolurl()[source]

Gets the pool URL of this pool.

getProductClasses()[source]

Returns all Product classes found in this pool. mh: returns an iterator.

getReferenceCount(ref)[source]

Returns the reference count of a ProductRef.

getScheme()[source]

Gets the scheme of this pool.

getUrnId()[source]

Get the identifier of this pool used to build URN, usually it’s same as id returned by getId().

isAlive()[source]

Test if the pool is capable of responding to commands.

isEmpty()[source]

Determines if the pool is empty.

loadDescriptors(urn)[source]

Loads the descriptors belonging to specified URN.

loadProduct(urn, serialize_out=False, asyn=False)[source]

Loads a Product belonging to specified URN.

serialize_out: if True returns contents in serialized form.

meta(urn)[source]

Loads the meta-data belonging to the product of specified URN.

property poolname

for property getter

property poolurl

for property getter

qm(qw, prod='BaseProduct', urns=None)[source]

short-hand method for select(qw, variable’m’, ptype=prod, previous=urns.

example: ..code: curl http://foo.edu:23456/data/pool/api/qm__m[“age”]>66 and m[“name”]==”Bob”’

reference(ref)[source]

Increment the reference count of a ProductRef.

remove(urn=None, resourcetype=None, index=None, ignore_error=False, asyn=False, **kwds)[source]

Removes a Product belonging to specified URN or a pair of data type and serial number.

removeAll(ignore_error=False, asyn=False, **kwds)[source]

Remove all pool data (self, products) and all pool meta data (self, descriptors, indices, etc.).

saveDescriptors(urn, desc)[source]

Save/Update descriptors in pool.

saveProduct(product, tag=None, geturnobjs=False, serialize_in=True, serialize_out=False, asyn=False, **kwds)[source]

Saves specified product(s) and returns the designated ProductRefs or URNs.

Saves a product or a list of products to the pool, possibly under the supplied tag(s), and returns the reference (or a list of references if the input is a list of products), or Urns if geturnobjs is True.

See pal document for pool structure.

Parameters:
  • product (BaseProduct, list) – Product or a list of them or ‘[ size1, prd, size2, prd2, …]’.

  • tag (str, list) – If given a tag, all products will be having this tag.

  • If a list tags are given to every one product then the

  • number of tags must not be the same to that of `product`. If

  • they are equal, each tag is goven to the product at the same

  • index in the `product` list.

  • serialize_out (bool) – if True returns contents in serialized form.

  • serialize_in (bool) – If set, product input is serialized.

schematicLoad(resourcetype, index, start=None, end=None, serialize_out=False)[source]

to be implemented by subclasses to do the scheme-specific loading

schematicRemove(urn=None, resourcetype=None, index=None, asyn=False, **kwds)[source]

to be implemented by subclasses to do the scheme-specific removing

schematicSave(products, tag=None, geturnobjs=False, serialize_in=True, serialize_out=False, asyn=False, **kwds)[source]

to be implemented by subclasses to do the scheme-specific saving

schematicSelect(query, previous=None)[source]

to be implemented by subclasses to do the scheme-specific querying.

schematicWipe()[source]

to be implemented by subclasses to do the scheme-specific wiping.

select(query, variable='m', ptype=<class 'fdi.dataset.product.Product'>, previous=None)[source]

Returns a list of references to products that match the specified query.

Parameters:
  • query (str) – the ‘where’ query string to make a query object.

  • variable (str) – name of the dummy variable in the query string. if variable is ‘m’, query goes via MetaQuery(ptype, query) ; else by AbstractQuery(ptype, variable, query) .

  • ptype (class) – The class object whose instances are to be queried. Or fragment of the name of such classes.

  • previous (list or str) – of urns, possibly from previous search. or a string of comma-separated urns, e.g. ‘urn:a:foo:12,urn:b:bar:9’

Returns:

of found URNs.

Return type:

list

setPoolManager(pm)[source]
setPoolname(poolname)[source]

Replaces the current poolname of this pool.

setPoolurl(poolurl)[source]

Replaces the current poolurl of this pool.

setup()[source]
Sets up interal machiney of this Pool,

but only if self._poolname and self._poolurl are present, and other pre-requisits are met.

Subclasses should implement own setup(), and make sure that self._poolname and self._poolurl are present with ``

if <pre-requisit not met>: return True if super().setup(): return True

# super().setup() has done its things by now. <do setup> return False

``

returns: True if not both self._poolname and self._poolurl are present.

static vectorize(*p)[source]
::

vectorize(9, [8,7,6]) -> ([9, 9, 9], [8, 7, 6], True)

fdi.pal.productpool.makeLockpath(direc, op='w')[source]

returns the appropriate path to put lock file.

creats the path if non-existing. Set lockpath-base permission to all-modify so other fdi users can use. op: ‘r’ for readlock no-reading) ‘w’ for writelock (no-writing)

fdi.pal.productpool.populate_pool2(tags, ptype, sn=None, dTypes=None, dTags=None)[source]
tagslist

The tags in a list. None and empty tags are ignored.

ptypestr

The product name / datatype / class name of the data item, new or existing.

snstr

Serial number. If is None, use the one in dTypes.

Returns:

dTypes and dTags with updates, and the index/serial number

Return type:

tuple

fdi.pal.productref module

class fdi.pal.productref.ProductRef(urn=None, poolname=None, product=None, meta=None, poolmanager=None, **kwds)[source]

Bases: MetaDataHolder, DeepcmpEqual, Serializable, Comparable

A lightweight reference to a product that is stored in a ProductPool or in memory.

Urn can be the string or URNobject.

Parameters:
  • urn (str)

  • poolname (str) – If given overrides the pool name in urn, and causes metadata to be loaded from pool, unless this prodref points to a mempool.

  • product (subclass of BaseProduct) – A productref created from a single product will result in a memory pool urn, and the metadata won’t be loaded.

  • meta (Metadata) – If meta is given, it will be used instead of that from poolname.

  • poolmanager (class) – subclass od PoolManager

  • **kwds

Returns:

Reference of a product.

Return type:

ProductRef

__init__(urn=None, poolname=None, product=None, meta=None, poolmanager=None, **kwds)[source]

Urn can be the string or URNobject.

Parameters:
  • urn (str)

  • poolname (str) – If given overrides the pool name in urn, and causes metadata to be loaded from pool, unless this prodref points to a mempool.

  • product (subclass of BaseProduct) – A productref created from a single product will result in a memory pool urn, and the metadata won’t be loaded.

  • meta (Metadata) – If meta is given, it will be used instead of that from poolname.

  • poolmanager (class) – subclass od PoolManager

  • **kwds

Returns:

Reference of a product.

Return type:

ProductRef

addParent(parent)[source]

add a parent

equals(o, verbose=False)[source]

true if o is a non-null ProductRef, with the same Product type than this one, and:

urns and products are null in both refs, or unrs are equal and products are null, or # <– mh urns are null in both refs, and their products are equal, or urns and products are equal in both refs

getHash()[source]

Returns a code number for the product; actually its MD5 signature. This allows checking whether a product already exists in a pool or not.

getMeta()[source]

Returns the metadata of the product.

getParents()[source]

Return the in-memory parent context products of this reference.

That is, the contexts in program memory that contain this product reference object. A context that contains a different product reference object pointing to the same URN is not a parent of this product reference.

Furthermore, it should be understood that this method does not return the parent contexts of the product pointed to by this reference as stored in any underlying pool or storage.

Returns: the parents

getPoolname()[source]

Returns the name of the product pool associated.

If not set, poolname from getUrnObj().getPoolname() is used and set to self.

getProduct()[source]

Get the product that this reference points to.

If the product is a Context, it is kept internally, so further accesses don’t need to ask the storage for loading it again. Otherwise, the product is returned but the internal reference remains null, so every call to this method involves a request to the storage.

This way, heavy products are not kept in memory after calling this method, thus maintaining the ProductRef a lighweight reference to the target product.

In case of a Context, if it is wanted to free the reference, call unload().

Returns: the product

getSize()[source]

Returns the estimated size(in bytes) of the product in memory.

Useful for providing this information for a user that wants to download the product from a remote site. Returns: the size in bytes

getStorage()[source]

Returns the product storage associated.

getType()[source]

Specifies the Product class to which this Product reference is pointing to.

getUrn()[source]

Returns the Uniform Resource Name (URN) of the product.

getUrnObj()[source]

Returns the URN as an object.

isLoaded()[source]

Informs whether the pointed product is already loaded.

property meta

Property

property parents

property

property product
removeParent(parent)[source]

remove a parent

Parameters:

parent

setParents(parents)[source]

Sets the in-memory parent context products of this reference.

Parameters:

parents

setStorage(storage)[source]

Sets the product storage associated.

setUrn(urn)[source]
setUrnObj(urnobj, poolname=None, meta=None)[source]

sets urn

A productref created from a single product will result in a memory pool urn, and the metadata won’t be loaded.

Parameters:

urnobjUrn

a URN object.

poolnamestr

if given overrides the pool name in urn, and causes metadata to be loaded from pool.

meta: MetaData

If is given, it will be used instead of that from poolname.

string(level=0, **kwds)
toString(level=0, **kwds)[source]
txt(level=0, **kwds)
unload()[source]

Clear the cached meta and frees internal reference to the product, so it can be garbage collected.

property urn

Property

property urnobj

Property

fdi.pal.productstorage module

class fdi.pal.productstorage.ProductStorage(pool=None, poolurl=None, poolmanager=None, **kwds)[source]

Bases: object

Logical store created from a pool or a poolURL.

Every instanciation with the same pool will result in a new instance of ProdStorage.

Gets the storage “control pannel” for pool with specifed name.

auth with client, can be given here to be passed to PoolManager.getPool. client how to call remote api if poolurl indicates a remote pool. Default is None for using the configured host/port/credentials. If doing a mocked server test, this needs to be set. :pool: can be given as the only pramemter. If auth and client are given they will substitute those of pool. If pool is not given, those will need to be given. :poolname: if is a string will be taken as a poolname. if is a pool object will be registered with its name, :poolurl: is sent to the PoolManager with poolname to get the pool object.

__init__(pool=None, poolurl=None, poolmanager=None, **kwds)[source]

Gets the storage “control pannel” for pool with specifed name.

auth with client, can be given here to be passed to PoolManager.getPool. client how to call remote api if poolurl indicates a remote pool. Default is None for using the configured host/port/credentials. If doing a mocked server test, this needs to be set. :pool: can be given as the only pramemter. If auth and client are given they will substitute those of pool. If pool is not given, those will need to be given. :poolname: if is a string will be taken as a poolname. if is a pool object will be registered with its name, :poolurl: is sent to the PoolManager with poolname to get the pool object.

accept(visitor)[source]

Hook for adding functionality to object through visitor pattern.

getAllTags()[source]

Get all tags defined in the writable pool.

getHead(ref)[source]

Returns the latest version of a given product, belonging to the first pool where the same track id is found.

getMeta(urn)[source]

Get the metadata belonging to the writable pool that associated to a given URN. returns an ODict.

getPool(poolname)[source]

mh: returns the pool object by poolname from this storage

getPools()[source]

Returns the set of ProductPools registered. mh: in a list of poolnames XXX TODO: getPoolnames

getProductClasses(poolname)[source]

Yields all Product classes found in this pool.

getTags(urn)[source]

Get the tags belonging to the writable pool that associated to a given URN. returns an iterator.

getUrnFromTag(tag)[source]

Get the URN belonging to the writable pool that is associated to a given tag.

getWritablePool(obj=False)[source]

returns the poolname of the first pool, which is the only writeable pool.

Obj:

(bool) return the pool objject instead of the name.

isEmpty()[source]

Returns whether all pools are empty or there is no pool.

load(urnortag)[source]

Loads a product with a URN or a list of products with a tag, from the (writeable) pool.

It always creates new ProductRefs. :return: productref if there is only one. A `list` of `ProductRefs`. urnortag: urn or tag

register(poolname=None, poolurl=None, pool=None, **kwds)[source]

Registers the given pools to the storage.

Client:

passed to PoolManager.getPool.

Auth:

passed to PoolManager.getPool.

remove(urn, datatype=None, index=None, ignore_error=False)[source]

removes product of urn from the writeable pool

save(product, tag=None, poolname=None, geturnobjs=False, asyn=False, **kwds)[source]

saves to the writable pool if it has been registered.

Parameters:
  • product (BaseProduct, list) – Product or a list of them or ‘[ size1, prd, size2, prd2, …]’.

  • tag (str, list) – If given a tag, all products will be having this tag.

  • If a list tags are given to every one product then the

  • number of tags must not be the same to that of `product`. If

  • they are equal, each tag is goven to the product at the same

  • index in the `product` list.

  • serialize_out (bool) – if True returns contents in serialized form.

  • serialize_in (bool) – If set, product input is serialized.

  • poolName (str) – If the named pool is not registered, registers and saves.

  • geturnobjs (bool) – returns UrnObjs if geturnobjs is True.

  • kwds (options passed to json.dump() for subclasses.)

Returns:

  • ProductRef (Product reference.)

  • Urn (If geturnobjs is set.)

  • str (If serialze_out is set, serialized form of ProductRef or URN.)

  • list (list of the above of input is a list.)

select(query, variable=None, ptype=None, previous=None)[source]

Returns a list of URNs to products that match the specified query.

Parameters:
  • query (the query object, or str) –

    The Query instances or

    the ‘where’ query string to make a query object.

  • variable (str) – name of the dummy variable in the query string. if variable is ‘m’, query goes via MetaQuery(ptype, query) ; else by AbstractQuery(ptype, variable, query) .

  • ptype (class) – The class object whose instances are to be queried. Or fragment of the name of such classes.

  • previous (list or str) – of urns, possibly from previous search. or a string of comma-separated urns, e.g. ‘urn:a:foo:12,urn:b:bar:9’

Return type:

the set of return eferences to products matching the supplied query.

unregister(pool=None, ignore_error=False, **kwds)[source]

Unregisters the given pools to the storage.

unregisterAll(ignore_error=False)[source]
wipePool(ignore_error=False, asyn=False, **kwds)[source]

Clear all data and meta data of the writable pool.

fdi.pal.publicclientpool module

fdi.pal.query module

class fdi.pal.query.AbstractQuery(product=<class 'fdi.dataset.product.Product'>, variable='p', where='', allVersions=False, **kwds)[source]

Bases: StorageQuery

provides default implementations for the pal storage query.

creates an AbstractQuery with product variable name, query string or function.

product: type (Class or ‘Card’ name) of the products to be queried. where: the query string.

__init__(product=<class 'fdi.dataset.product.Product'>, variable='p', where='', allVersions=False, **kwds)[source]

creates an AbstractQuery with product variable name, query string or function.

product: type (Class or ‘Card’ name) of the products to be queried. where: the query string.

class fdi.pal.query.MetaQuery(product=<class 'fdi.dataset.product.Product'>, where='', allVersions=False, **kwds)[source]

Bases: AbstractQuery

Meta data query formulates a query on the meta data of a Product.

Typically this type of query is faster than a full query on the Product Access Layer.

creates an MetaQuery with a query string or function.

product: type (Class or ‘Card’ name) of the products to be queried. where: the query string. ‘where’ is a query string or function that returns True or False. In the query string variable name is ‘m’ for a MetaData type, as in m = product.meta.

__init__(product=<class 'fdi.dataset.product.Product'>, where='', allVersions=False, **kwds)[source]

creates an MetaQuery with a query string or function.

product: type (Class or ‘Card’ name) of the products to be queried. where: the query string. ‘where’ is a query string or function that returns True or False. In the query string variable name is ‘m’ for a MetaData type, as in m = product.meta.

class fdi.pal.query.StorageQuery(**kwds)[source]

Bases: Serializable

Query on a ProductStorage.

accept(visitor)[source]

Hook for adding functionality to object through visitor pattern.

property allVersions
getAllVersions()[source]

Are all versions to be retrieved, or just the latest?

getType()[source]

Get the class used in the query.

getVariable()[source]

Get the variable name used in the query expression, eg “p”.

getWhere()[source]

Get the query expression to be evaluated.

retrieveAllVersions()[source]

Returns the allVersions related to this object.

setAllVersions(allVersions)[source]

Sets the allVersions of this object.

setType(type)[source]

Sets the type of this object.

setVariable(variable)[source]

Sets the variable of this object.

setWhere(where)[source]

Sets the where of this object.

string(level=0, tablefmt='rst', tablefmt1='simple', tablefmt2='rst', **kwds)
toString(level=0, tablefmt='rst', tablefmt1='simple', tablefmt2='rst', **kwds)[source]
txt(level=0, tablefmt='rst', tablefmt1='simple', tablefmt2='rst', **kwds)
property type
property variable
property where

fdi.pal.taggable module

class fdi.pal.taggable.Taggable(**kwds)[source]

Bases: object

Definition of services provided by a product storage supporting tagging.

getTagUrnMap()[source]

Get the full tag->urn mappings.

mh: returns an iterator

getTags(urn=None, datatype=None, sn=None)[source]

Get all of the tags that map to a given URN or a pair of data type and serial number.

Get all known tags if urn is not specified. mh: returns an iterator.

If datatype and sn are given, use them and ignore urn.

getUrn(tag)[source]

Gets the URNs corresponding to the given tag. Returns an empty list if tag does not exist.

getUrnObject(tag)[source]

Gets the URNobjects corresponding to the given tag.

removeTag(tag)[source]

Remove the given tag from the tag and urn maps.

removeUrn(urn=None, datatype=None, sn=None)[source]

Remove the given urn from the tag and urn maps.

setTag(tag, urn=None, datatype=None, sn=None)[source]

Sets the specified tag to the given URN.

tagExists(tag)[source]

Tests if a tag exists.

fdi.pal.urn module

class fdi.pal.urn.Urn(urn=None, poolname=None, cls=None, index=None, poolurl=None, **kwds)[source]

Bases: DeepcmpEqual, Serializable, Comparable

The object representation of the product URN string.

The memory consumed by sets of this object are much less than sets of URN strings.

Only when the class types in URN string are not in classpath, the urn object will consume equals or a little more than URN string as the object has to hold the original urn string. However this should be considered as exceptional cases.

Using this object representation also help to avoid parsing cost of URN string. URN string should be immutable.

About_URN

The Universial Resource Name (URN, https://datatracker.ietf.org/doc/html/rfc2141 ) string has this format:

urn:<poolname>:<resourcetype>:<serialnumber>

with modified rules desribed below.

<poolname>:

Also called poolID. It consists of 1-32 characters, is case-sensitive, which deviates from rfc2141. Character allowed are alpha, digit, safe, defined in rfc1630 (https://datatracker.ietf.org/doc/html/rfc1630). These are excluded: `` , ``%, ?, !, *,``’, ``", (, ), =, /, and what listed in mod:poolmanager:Invalid_Pool_Names, e.g. pools, urn, URN, api.

<resourcetype>:

type name of the data item (usually class name of data products inheriting BaseProduct)

<serialnumber>:

internal index for a certain <resourcetype>.

The poolname in a URN is a label. Some examples:

URNs are used to to identify data be cause URNs are location agnostic. Storage Pools (subclasses of ProductPool) are where data item reside. The PoolURL is used to give practical information of a pool, such as a poolname, its location, and its access scheme. PoolURL is designed to be a local set-up detail that is supposed to be hidden from pool users. Data processing software use ``URN``s to refer to products, without specifying pool location. The poolID in a URN could be a LocalPool on the development laptop and a HTTPClientPool on the production cloud.

Creates the URN object with the urn string or components.

give urn and optional poolurl, or all poolname, cls, index arguments. if urn is given and pool, class, etc are also specified, the latter are ignored. else the URN object is constructed from them. Urn(u) will make a Urn object out of u.

All arguements are None by default.

Parameters urn: string. A URM string. poolname: string, provides pool name part of URN if URN is missing from input. cls: type. the full class name is used for datatype. index: int. poolurl: string. IF specified will provide info of the pool involved.

__init__(urn=None, poolname=None, cls=None, index=None, poolurl=None, **kwds)[source]

Creates the URN object with the urn string or components.

give urn and optional poolurl, or all poolname, cls, index arguments. if urn is given and pool, class, etc are also specified, the latter are ignored. else the URN object is constructed from them. Urn(u) will make a Urn object out of u.

All arguements are None by default.

Parameters urn: string. A URM string. poolname: string, provides pool name part of URN if URN is missing from input. cls: type. the full class name is used for datatype. index: int. poolurl: string. IF specified will provide info of the pool involved.

getIndex()[source]

Returns the product index.

getPlace()[source]

Returns the netloc in this

getPool()[source]

Returns the pool name in this

getPoolId()[source]

Returns the pool URN in this

getPoolpath()[source]

returns the poolpath stored

getScheme()[source]

Returns the urn scheme.

getType()[source]

Returns class type of Urn

getTypeName()[source]

Returns class type name of Urn.

getUrn()[source]

Returns the urn in this

getUrnWithoutPoolId()[source]
property place
property pool

returns the poolname.

setUrn(urn, poolurl=None)[source]

parse urn to get poolname, resource, index.

string(level=0, **kwds)
toString(level=0, **kwds)[source]
txt(level=0, **kwds)
property urn

property

class fdi.pal.urn.UrnUtils[source]

Bases: object

static checkUrn(identifier)[source]

Throw a ValueError if the identifier is not a legal URN.

static containsUrn(poolobj, urn)[source]

Informs whether a URN belongs to the given pool.

static extractRecordIDs(urns)[source]

Extracts product IDs (serial numbers) from a set of urns.

static getClass(urn)[source]

Get the class contained in a URN.

static getClassName(urn)[source]

Get the class name contained in a URN.

static getLater(urn1, urn2)[source]

Returns the later of two urns.

static getPool(urn, pools)[source]

Returns the pool corresponding to the pool id inside the given urn.

pools: ProductPool or subclass

static getPoolId(urn)[source]

Returns the pool id part of the URN.

static getProductId(urn)[source]

Returns the product id part of the URN, that is, the last token.

static isUrn(identifier)[source]

Informs whether the given identifier corresponds to a URN.

fdi.pal.urn.is_urn(u)[source]
fdi.pal.urn.makeUrn(poolname, typename, index)[source]

assembles a URN or a list of URNs with infos of the pool, the resource type, and the index.

Poolname:

str or list of them.

Typename:

str, name of data type. or list of them.

index: int or string or list of them. If any two or more are lists, all have to be list of the same length.

fdi.pal.urn.parseUrn(urn, int_index=True, check_poolename=None)[source]

Checks the URN string is valid in its form and splits it.

A Product URN has several segment. For example if the urn is urn:mypool/v2:proj1.product:322 * poolname, also called poolURN or poolID, optionally path-like: mypool/v2, * resource type (usually class) name proj1.product, * index number 322, If urn is None or empty returns (None,None,None)

Parameter

urnstr,list

One or a list of URN strings to be decomposed.

int_indexbool

If True (default) returns integer index, else string index.

check_poolenamestr

Raise ValueError is any urn has a poolname different from the value of check_poolename.

returns:
  • tuple

  • * If urn is None or a zero-length string, returns (None, None, None).

  • * If urn is a non-zero-length string, returns a tuple of – :poolname: Name of the pool :resourceclass: type of resource/products :index: One or a list of (int) serial number of resourceclass in the pool.

  • * If urn is a list URNs and all of them have identical poolname and identical resourceclasses, returns a tuple (poolname, resourceclass, list-of-index])

  • * If urn is a list URNs and not all of them have identical poolname or not identical resourceclasses, returns a tuple (list_of_poolname, list_of_resourceclass, list-of-index), …)

fdi.pal.urn.parse_poolurl(url, poolhint=None)[source]

Disassambles a pool URL.

A Pool URL is It is generated to desribe . For example:

input: * url: to be decomposed. * poolhint: A urn or a poolname (the first distinctive substring) needs to be given if the poolname has more than one level.

returns: poolpath, scheme, place, poolname. returns (None, None,None,None) if url is None or empty.

About_poolURL

The PoolURL format is in the form of a URL that preceeds its poolname part:

<scheme>://<place><poolpath>/<poolname>

<scheme>:

Implementation protocol including file for LocalPool, mem for MemPool, http, https for HttpclientPool.

<place>:

IP:port such as``192.168.5.6:8080`` for http and https schemes, or an empty string for file and mem schemes.

<poolname>:

same as in URN.

<poolpath>:

The part between place and an optional poolhint:

<username>:

<password>:

  • For file or server schemes, e.g. poolpath is /c:/tmp in http://localhost:9000/c:/tmp/mypool/ with poolhint keyword arguement of parse_poolurl() not given, or given as mypool (or myp or my …).

  • For http and https schemes, it is e.g. /0.6/tmp in https://10.0.0.114:5000/v0.6/tmp/mypool with poolhint keyword arguement not given, or given as mypool (or myp` or 'my' ...). The meaning of poolpath is subject to interpretation by the  server. In the preceeding example the poolpath has an API version.  :meth:`ProductPool.transformpath` is used to map it further. Note that trailing blank and ``/ are ignored, and stripped in the output.

Examples:

fdi.pal.versionable module

class fdi.pal.versionable.Versionable(**kwds)[source]

Bases: object

for items being able to be defined with a Definition.

getLastVersion(ref)[source]

Returns the latest version of the given ProductRef.

getVersions(ref)[source]

Returns all the versions of the given ProductRef.

saveProductRef(ref)[source]

Saves the product referenced and returns the designated URN.

fdi.pal.webapi module