fdi.dataset package

Submodules

fdi.dataset.abstractcomposite module

class fdi.dataset.abstractcomposite.AbstractComposite(**kwds)

Bases: Attributable, Annotatable, Composite, DataWrapperMapper, DatasetListener

an annotatable and attributable subclass of Composite.

toString(matprint=None, trans=True, beforedata='')

fdi.dataset.annotatable module

class fdi.dataset.annotatable.Annotatable(description='UNKNOWN', **kwds)

Bases: object

An Annotatable object is an object that can give a human readable description of itself.

a__init__(*args, description='UNKNOWN', **kwds)

getDescription(): gets the description of this Annotatable object.

setDescription(newDescription): sets the description of this Annotatable object.

fdi.dataset.attributable module

class fdi.dataset.attributable.Attributable(meta=None, **kwds)

Bases: MetaDataHolder

An Attributable object is an object that has the notion of meta data.

property meta

setMeta(newMetadata): Replaces the current MetaData with specified argument. mh: Product will override this to add listener whenevery meta is replaced

fdi.dataset.baseproduct module

class fdi.dataset.baseproduct.BaseProduct(description='UNKOWN', type_='BaseProduct', creator='UNKOWN', creationDate=2017-01-01T00:00:00.000000 TAI(0), rootCause='UNKOWN', schema='0.3', **kwds)

Bases: AbstractComposite, Copyable, Serializable, EventSender

A BaseProduct is a generic result that can be passed on between (standalone) processes.

In general a Product contains zero or more datasets, history, optional metadata as well as some required metadata fields. Its intent is that it can fully describe itself; this includes the way how this product was achieved (its history). As it is the result of a process, it should be able to save to and restore from an Archive device.

Many times a Product may contain a single dataset and for this purpose the first dataset entry can be accessed by the getDefault() method. Note that the datasets may be a composite of datasets by themselves.

mh: Built-in Attributes in productInfo[‘metadata’] can be accessed with e.g. p.creator or p.meta[‘description’].value: p.creator=’foo’ assert p.creatur==’foo’ assert p.meta[‘creator’]==’foo’ p.meta[‘creator’]=Parameter(‘bar’) assert p.meta[‘creator’]==Parameter(‘bar’)

accept(visitor): Hook for adding functionality to meta data object through visitor pattern.

getDefault(): Convenience method that returns the first dataset belonging to this product.

installMetas(mtbi): put parameters in group in product metadata, and updates productInfo. values in mtbi override those default ones in group.

productInfo = {'metadata': {'creationDate': {'data_type': 'finetime', 'default': '0', 'description': 'Creation date of this product', 'fits_keyword': 'DATE', 'unit': 'None'}, 'creator': {'data_type': 'string', 'default': 'UNKOWN', 'description': 'Generator of this product. Example name of institute, organization, person, software, special algorithm etc.', 'fits_keyword': 'CREATOR', 'unit': 'None'}, 'description': {'data_type': 'string', 'default': 'UNKOWN', 'description': 'Description of this product', 'fits_keyword': 'DESCRIPT', 'unit': 'None'}, 'rootCause': {'data_type': 'string', 'default': 'UNKOWN', 'description': 'Reason of this run of pipeline.', 'fits_keyword': 'ROOTCAUS', 'unit': 'None'}, 'schema': {'data_type': 'string', 'default': '0.3', 'description': 'Version of product schema', 'fits_keyword': 'SCHEMA', 'unit': 'None'}, 'type': {'data_type': 'string', 'default': 'BaseProduct', 'description': 'Product Type identification. Fully qualified Python class name or CARD.', 'fits_keyword': 'TYPE', 'unit': 'None'}}}

serializable(): Can be encoded with serializableEncoder

setMeta(newMetadata): Replaces the current MetaData with specified argument. mh: Product will override this to add listener whenevery meta is replaced

targetChanged(event): Informs that an event has happened in a target of the specified type.

toString(matprint=None, trans=True, beforedata=''): like AbstractComposite but with history

class fdi.dataset.baseproduct.History(other=None, **kwds)

Bases: CompositeDataset, DeepEqual

Public interface to the history dataset. Contains the main methods for retrieving a script and copying the history.

accept(visitor): Hook for adding functionality to meta data object through visitor pattern.

getOutputVar(): Returns the final output variable of the history script.

getScript(): Creates a Jython script from the history.

getTaskHistory(): Returns a human readable formatted history tree.

saveScript(file): Saves the history script to a file.

serializable(): Can be encoded with serializableEncoder

fdi.dataset.baseproduct.addMandatoryProductAttrs(cls): mh: Add MPAs to a class so that although they are metadata, they can be accessed by for example, productfoo.creator. dynamic properties see https://stackoverflow.com/a/2584050 https://stackoverflow.com/a/1355444

fdi.dataset.classes module

class fdi.dataset.classes.Classes

Bases: object

A dictionary of class names and their class objects that are allowed to be deserialized. A fdi package built-in dictionary (in the format of locals() output) is kept internally. Users who need add more deserializable class can for example: def myclasses():

from foo.bar import Baz …

Classes.classes = myClasses

class fdi.dataset.classes.Classes_meta(*args, **kwds)

Bases: type

metaclass for ‘classproperty’. # https://stackoverflow.com/a/1800999

makePackageClasses(): The set of fdi package-wide deserializable classes is maintained by hand. Do nothing if the classes mapping is already made so repeated calls will not cost lots more time.

property mapping: Returns the dictionary of classes allowed for deserialization, including the fdi built-ins and user added classes.

updateMapping(c={}): Updates classes mapping. Make the package mapping if it has not been made.

fdi.dataset.classes.logger = <Logger fdi.dataset.classes (WARNING)>: Note: this has to be in a different file where other interface classes are defined to avoid circular dependency (such as , Serializable.

fdi.dataset.collectionsMockUp module

class fdi.dataset.collectionsMockUp.ContainerMockUp: Bases: object

class fdi.dataset.collectionsMockUp.MappingMockUp: Bases: object

class fdi.dataset.collectionsMockUp.SequenceMockUp: Bases: object

fdi.dataset.composite module

class fdi.dataset.composite.Composite(**kwds)

Bases: DeepEqual

A container of named Datasets.

This container can hold zero or more datasets, each of them stored against a unique name. The order of adding datasets to this Composite is important, that is: the keySet() method will return a set of labels of the datasets in the sequence as they were added. Note that replacing a dataset with the same name, will keep the order.

containsKey(name): Returns true if this map contains a mapping for the specified name.

get(name): Returns the dataset to which this composite maps the specified name. If the attitute does not exist, return None. This is an OrderedDict behavior.

getSets(): Provide access to the Map < String, Dataset > . mh: api from CompositeDataset

isEmpty(): Returns true if this map contains no key - value mappings.

items(): Enable pairs = [(v, k) for (k, v) in d.items()].

keySet(): Returns a set view of the keys contained in this composite.

remove(name): Removes the mapping for this name from this composite. mh: returns None if name is None or item does not exist.

set(name, dataset): Associates the specified dataset with the specified key in this map(optional operation). If the map previously contained a mapping for this key, the old dataset is replaced by the specified dataset. this composite does not permit null or empty keys.

size(): Returns the number of key - value mappings in this map.

toString(matprint=None, trans=True)

values(): Enable pairs = zip(d.values(), d.keys())

fdi.dataset.copyable module

class fdi.dataset.copyable.Copyable

Bases: object

Interface for objects that can make a copy of themselves.

copy(): Makes a deep copy of itself.

fdi.dataset.dataset module

class fdi.dataset.dataset.ArrayDataset(*args, **kwds)

Bases: DataWrapper, GenericDataset, Sequence

Special dataset that contains a single Array Data object. mh: If omit the parameter names during instanciation, e.g. ArrayDataset(a, b, c), the assumed order is data, unit, description. mh: contains a sequence which provides methods count(), index(), remove(), reverse(). A mutable sequence would also need append(), extend(), insert(), pop() and sort().

append(*args, **kwargs): appends to data.

count(*args, **kwargs): returns size.

index(*args, **kwargs): returns the index of a value.

pop(*args, **kwargs): revomes and returns value

remove(*args, **kwargs): removes value at first occurrence.

serializable(): Can be encoded with serializableEncoder

setData(data)

toString(matprint=None, trans=True): matprint: an external matrix print function trans: print 2D matrix transposed. default is True.

class fdi.dataset.dataset.Column(*args, **kwds)

Bases: ArrayDataset, ColumnListener

A Column is a the vertical cut of a table for which all cells have the same signature. It contains raw ArrayData, and optionally a description and unit. example:

table = TableDataset()
table.addColumn("Energy",Column(data=[1,2,3,4],description="desc",unit='eV'))

class fdi.dataset.dataset.CompositeDataset(**kwds)

Bases: AbstractComposite, Dataset

An CompositeDataset is a Dataset that contains zero or more named Datasets. It allows to build arbitrary complex dataset structures.

It also defines the iteration ordering of its children, which is the order in which the children were inserted into this dataset.

serializable(): Can be encoded with serializableEncoder

class fdi.dataset.dataset.Dataset(**kwds)

Bases: Attributable, Annotatable, Copyable, Serializable, DeepEqual, MetaDataListener

Attributable and annotatable information data container that can be be part of a Product.

developer notes: The intent is that developers do not derive from this interface

directly. Instead, they should inherit from one of the generic datasets that this package provides:

mh: GenericDataset, ArrayDataset. TableDataset or CompositeDataset.

accept(visitor): Hook for adding functionality to object through visitor pattern.

toString()

class fdi.dataset.dataset.GenericDataset(**kwds)

Bases: Dataset, DataContainer, Container

mh: Contains one data item.

serializable(): Can be encoded with serializableEncoder

toString(matprint=None, trans=True): matprint: an external matrix print function trans: print 2D matrix transposed. default is True.

class fdi.dataset.dataset.TableDataset(**kwds)

Bases: Dataset, TableModel

Special dataset that contains a single Array Data object. A TableDataset is a tabular collection of Columns. It is optimized to work on array data.. The column-wise approach is convenient in many cases. For example, one has an event list, and each algorithm is adding a new field to the events (i.e. a new column, for example a quality mask).

Although mechanisms are provided to grow the table row-wise, one should use these with care especially in performance driven environments as this orthogonal approach (adding rows rather than adding columns) is expensive.

General Note:

For reasons of flexibility, memory consumption and performance, this class is not checking whether all columns are of the same length: this is the responsibility of the user/developer. See also the library documentation for more information about this.

Note on column names:

If a column is added without specifying a name, the name ColumnX is created, where X denotes the index of that column. Column name duplicity is not allowed.

Developers:

See “Writing special datasets or products” at the developer’s documentation also.

Please see also this selection example.

addColumn(name, column)

Adds the specified column to this table, and attaches a name to it. If the name is null, a dummy name is created, such that it can be accessed by getColumn(str).

Duplicated column names are not allowed.

Parameters: name - column name. column - column to be added.

addRow(row): Adds the specified map as a new row to this table. mh: row is a dict with names as keys

property columnCount

getColumn(key): return colmn if given string as name or int as index. returns name if given column.

getRow(rowIndex): Returns a list containing the objects located at a particular row.

indexOf(key): Returns the index of specified column; if the key is a Column, it looks for equal references (same column objects), not for equal values. If the key is a string, Returns the index of specified Column name. mh: Or else returns the key itself.

removeColumn(key)

removeRow(rowIndex): Removes a row with specified index from this table. mh: returns removed row.

property rowCount

select(selection): Select a number of rows from this table dataset and return a new TableDataset object containing only the selected rows.

serializable(): Can be encoded with serializableEncoder

setColumn(key, value): Replaces a column in this table with specified name to specified column if key exists, or else add a new coolumn.

setColumnCount(columnCount): cannot do this.

setData(data): sets name-column pairs from [{‘name’:str,’column’:Column}] or {str:Column} or [[num]] or [(str, [], ‘str’)] form of data, Existing data will be discarded except when the provided data is a list of lists, where existing column names and units will remain but data replaced, and extra data items will form new columns named ‘col[index]’ (index counting from 1) with unit None.

setRowCount(rowCount): cannot do this.

toString(matprint=None, trans=True)

class fdi.dataset.dataset.TableModel(**kwds)

Bases: DataContainer

to interrogate a tabular data model

col(columIndex): returns a tuple of (name, column) at the column index.

getColumnClass(columnIndex): Returns the most specific superclass for all the cell values in the column.

getColumnCount(): Returns the number of columns in the model.

getColumnName(columnIndex): Returns the name of the column at columnIndex.

getRowCount(): Returns the number of rows in the model.

getValueAt(rowIndex, columnIndex): Returns the value for the cell at columnIndex and rowIndex.

isCellEdidata(rowIndex, columnIndex): Returns true if the cell at rowIndex and columnIndex is edidata.

setValueAt(aValue, rowIndex, columnIndex): Sets the value in the cell at columnIndex and rowIndex to aValue.

fdi.dataset.datatypes module

class fdi.dataset.datatypes.Quaternion(components=[0, 0, 0, 0], **kwds)

Bases: Vector

Quaternion with a unit.

class fdi.dataset.datatypes.Vector(components=[0, 0, 0], description='UNKNOWN', unit='', **kwds)

Bases: Annotatable, Copyable, DeepEqual, Quantifiable, Serializable

Three dimensional vector with a unit.

accept(visitor): Adds functionality to classes of this components.

property components: for property getter

getComponents(): Returns the actual components that is allowed for the components of this vector.

serializable(): Can be encoded with serializableEncoder

setComponents(components): Replaces the current components of this vector.

fdi.dataset.datawrapper module

class fdi.dataset.datawrapper.DataContainer(data=None, **kwds)

Bases: Annotatable, Quantifiable, Copyable, DeepEqual

A DataContainer is a composite of data and description. mh: note that There is no metadata. Implemented partly from AbstractDataWrapper.

property data

getData(): Returns the data in this dw

hasData(): Returns whether this data wrapper has data.

setData(data): Replaces the current DataData with specified argument. mh: subclasses can override this to add listener whenevery data is replaced

class fdi.dataset.datawrapper.DataWrapper(data=None, **kwds)

Bases: DataContainer

A DataWrapper is a composite of data, unit and description. mh: note that all data are in the same unit. There is no metadata. Implemented from AbstractDataWrapper.

class fdi.dataset.datawrapper.DataWrapperMapper

Bases: object

Object holding a map of data wrappers.

getDataWrappers(): Gives the data wrappers, mapped by name.

fdi.dataset.deserialize module

class fdi.dataset.deserialize.IntDecoder(*, object_hook=None, parse_float=None, parse_int=None, parse_constant=None, strict=True, object_pairs_hook=None)

Bases: JSONDecoder

adapted from https://stackoverflow.com/questions/45068797/how-to-convert-string-int-json-into-real-int-with-json-loads modified to also convert keys in dictionaries.

decode(s): Return the Python representation of s (a str instance containing a JSON document).

fdi.dataset.deserialize.constructSerializableClassID(obj, lgb=None, debug=False): mh: reconstruct object from the output of jason.loads(). Recursively goes into nested class instances that are not encoded by default by JSONEncoder, instantiate and fill in variables. Objects to be deserialized must have their classes loaded. ClassID cannot have module names in it (e.g. dataset.Product) or locals()[classname] or globals()[classname] will not work. See alternative in https://stackoverflow.com/questions/452969/does-python-have-an-equivalent-to-java-class-forname

fdi.dataset.deserialize.deserializeClassID(js, lgb=None, debug=False, usedict=False)

Loads classes with ClassID from the results of serializeClassID.

if usedict is True dict insted of ODict will be used.

fdi.dataset.deserialize.imakedesables(): makes a class dictionary for instanciation.

fdi.dataset.deserialize.lls(s, length=80): length-limited string

fdi.dataset.deserialize.logger = <Logger fdi.dataset.deserialize (WARNING)>: Note: this has to be in a different file where other interface classes are defined to avoid circular dependency (such as , Serializable.

fdi.dataset.eq module

class fdi.dataset.eq.DeepEqual

Bases: object

mh: Can compare key-val pairs of another object with self. False if compare with None or exceptions raised, e.g. obj does not have items()

diff(obj, seenlist): recursively compare components of list and dict. until meeting equality. seenlist: a list of classes that has been seen. will not descend in to them.

equals(obj)

class fdi.dataset.eq.EqualDict

Bases: object

mh: Can compare key-val pairs of another object with self. False if compare with None or exceptions raised, e.g. obj does not have items()

equals(obj)

class fdi.dataset.eq.EqualODict

Bases: object

mh: Can compare order and key-val pairs of another object with self. False if compare with None or exceptions raised, e.g. obj does not have items()

equals(obj)

fdi.dataset.eq.deepcmp(obj1, obj2, seenlist=None, verbose=False): Recursively descends into obj1’s every member, which may be set, list, dict, ordereddict, (or ordereddict subclasses) and any objects with ‘__class__’ attribute, compares every member found with its counterpart in obj2. Returns None if finds no difference, a string of explanation otherwise. Detects cyclic references.

fdi.dataset.finetime module

class fdi.dataset.finetime.FineTime(date=None, **kwds)

Bases: Copyable, DeepEqual, Serializable

Atomic time(SI seconds) elapsed since the TAI epoch of 1 January 1958 UT2. The resolution is one microsecond and the allowable range is: epoch + /-290, 000 years approximately.

This has the following advantages, compared with the standard class:

It has better resolution(microseconds) Time differences are correct across leap seconds It is immutable

EPOCH = datetime.datetime(1958, 1, 1, 0, 0, tzinfo=<fdi.dataset.finetime.UTC object>)

RESOLUTION = 1000000

classmethod datetimeToFineTime(dtm): Return given Python Datetime as a FineTime.

equals(obj): can compare TAI directly

microsecondsSinceEPOCH(): Return the rounded integer number of microseconds since the epoch: 1 Jan 1958.

serializable(): Can be encoded with serializableEncoder

subtract(time): Subract the specified time and return the difference in microseconds.

toDate(): Return this time as a Python Datetime.

classmethod toDatetime(tai): Return given FineTime as a Python Datetime.

toString(): Returns a String representation of this object. prints like 2019 - 02 - 17T12: 43: 04.577000 TAI

class fdi.dataset.finetime.FineTime1(date=None, **kwds)

Bases: FineTime

Same as FineTime but Epoch is 2017-1-1 0 UTC

EPOCH = datetime.datetime(2017, 1, 1, 0, 0, tzinfo=<fdi.dataset.finetime.UTC object>)

RESOLUTION = 1000

class fdi.dataset.finetime.UTC

Bases: tzinfo

https://docs.python.org/2.7/library/datetime.html?highlight=datetime#datetime.tzinfo

HOUR = datetime.timedelta(0, 3600)

ZERO = datetime.timedelta(0)

dst(dt): datetime -> DST offset in minutes east of UTC.

tzname(dt): datetime -> string name of time zone.

utcoffset(dt): datetime -> timedelta showing offset from UTC, negative values indicating West of UTC

fdi.dataset.listener module

class fdi.dataset.listener.ColumnListener

Bases: DatasetBaseListener

Listener for events occuring in a Column.

Available types:

* DESCRIPTION_CHANGED
* UNIT_CHANGED
* DATA_CHANGED

Cause is always null.

class fdi.dataset.listener.DatasetBaseListener

Bases: EventListener

Generic interface for listeners that will listen to events happening on a target of a specific type. Java Warning: The listener must be a class field in order to make an object hard reference.

targetChanged(event): Informs that an event has happened in a target of the specified type.

class fdi.dataset.listener.DatasetEvent(source, target, type_, change, cause, rootCause, **kwds)

Bases: Serializable

serializable(): Can be encoded with serializableEncoder

toString()

class fdi.dataset.listener.DatasetEventSender(**kwds)

Bases: EventSender

addListener(listener, cls=<class 'fdi.dataset.listener.DatasetBaseListener'>): Adds a listener to this.

fire(event)

class fdi.dataset.listener.DatasetListener

Bases: DatasetBaseListener

Listener for events occuring in MetaData. Available types:

* DESCRIPTION_CHANGED, METADATA_CHANGED (all datasets)
* DATA_CHANGED, UNIT_CHANGED (ArrayDataset)
* COLUMN_ADDED, COLUMN_REMOVED, COLUMN_CHANGED, ROW_ADDED, VALUE_CHANGED (TableDataset)
* DATASET_ADDED, DATASET_REMOVED, DATASET_CHANGED (CompositeDataset)

Possible causes:

* not null (METADATA_CHANGED, COLUMN_CHANGED, DATASET_CHANGED)
* null (rest)

Warning: The listener handler must be a class attribute in order to create an object hard reference. See DatasetBaseListener.

class fdi.dataset.listener.EventListener

Bases: object

Generic interface for listeners that will listen to anything

targetChanged(*args, **kwargs): Informs that an event has happened in a target of any type.

class fdi.dataset.listener.EventSender(**kwds)

Bases: object

adapted from Peter Thatcher’s https://stackoverflow.com/questions/1092531/event-system-in-python/1096614#1096614

addListener(listener, cls=<class 'fdi.dataset.listener.EventListener'>): Adds a listener to this.

fire(*args, **kwargs)

getListenerCount()

getListeners(): Returns the current Listeners.

property listeners

removeListener(listener): Removes a listener from this.

setListeners(listeners): Replaces the current Listeners with specified argument.

class fdi.dataset.listener.EventType

Bases: object

COLUMN_ADDED = 0

COLUMN_CHANGED = 1

COLUMN_REMOVED = 2

DATASET_ADDED = 4

DATASET_CHANGED = 5

DATASET_REMOVED = 6

DATA_CHANGED = 3

DESCRIPTION_CHANGED = 7

METADATA_CHANGED = 8

PARAMETER_ADDED = 9

PARAMETER_CHANGED = 10

PARAMETER_REMOVED = 11

ROW_ADDED = 12

ROW_REMOVED = 13

UNIT_CHANGED = 14

VALUE_CHANGED = 15

class fdi.dataset.listener.ListnerSet(**kwds)

Bases: Serializable, DeepEqual, list

Mutable collection of Listeners of an EvenSender.

equals(obj): compares with another one.

geturns(remove=None): Returns the current urns.

serializable(): Can be encoded with serializableEncoder

seturns(urns): Replaces the current urn with specified argument.

property urns

class fdi.dataset.listener.MetaDataListener

Bases: DatasetBaseListener

Listener for events occuring in MetaData. Available types:

* PARAMETER_ADDED
* PARAMETER_REMOVED
* PARAMETER_CHANGED

Possible causes: not null (for PARAMETER_CHANGED, if parameter internally changed) null (for PARAMETER_CHANGED, when set is called with a previous existing parameter, and rest)

Warning: The listener handler must be a class attribute in order to create an object hard reference. See DatasetBaseListener.

class fdi.dataset.listener.ParameterListener

Bases: DatasetBaseListener

Listener for events occuring in a Parameter. Available types:

* DESCRIPTION_CHANGED
* UNIT_CHANGED
* VALUE_CHANGED

Cause is always null.

Warning: The listener handler must be a class attribute in order to create an object hard reference. See DatasetBaseListener.

class fdi.dataset.listener.ProductListener

Bases: DatasetBaseListener

Listener for events occuring in Product. Available types:

* METADATA_CHANGED
* DATASET_ADDED
* DATASET_REMOVED
* DATASET_CHANGED

Possible causes:

* not null (METADATA_CHANGED, DATASET_CHANGED)
* null (METADATA_CHANGED, DATASET_REMOVED, DATASET_CHANGED)

Warning: The listener handler must be a class attribute in order to create an object hard reference. See DatasetBaseListener.

fdi.dataset.metadata module

class fdi.dataset.metadata.MetaData(copy=None, **kwds)

Bases: Composite, Copyable, Serializable, ParameterListener, DatasetEventSender

A container of named Parameters. A MetaData object can have one or more parameters, each of them stored against a unique name. The order of adding parameters to this container is important, that is: the keySet() method will return a set of labels of the parameters in the sequence as they were added. Note that replacing a parameter with the same name, will keep the order.

accept(visitor): Hook for adding functionality to meta data object through visitor pattern.

clear(): Removes all the key - parameter mappings.

remove(name): add eventhandling

serializable(): Can be encoded with serializableEncoder

set(name, newParameter): Saves the parameter and add eventhandling. Raises TypeError if not given Parameter (sub) class object.

toString()

class fdi.dataset.metadata.NumericParameter(**kwds)

Bases: Parameter, Quantifiable

has a number as the value and a unit.

serializable(): Can be encoded with serializableEncoder

class fdi.dataset.metadata.Parameter(value=None, description='UNKNOWN', type_='', **kwds)

Bases: Annotatable, Copyable, DeepEqual, DatasetEventSender, Serializable

Parameter is the interface for all named attributes in the MetaData container. It can have a value and a description.

accept(visitor): Adds functionality to classes of this type.

equals(obj): can compare value

getType(): Returns the actual type that is allowed for the value of this Parameter.

getValue(): Gets the value of this parameter as an Object.

serializable(): Can be encoded with serializableEncoder

setType(type_): Replaces the current type of this parameter. Unsuported parameter types will get a NotImplementedError.

setValue(value): Replaces the current value of this parameter. If given/current type_ is ‘’ and arg value’s type is in ParameterTypes both value and type are updated to the suitable one in ParameterDataTypes; or else TypeError is raised. If value type and given/current type_ are different.

Incompatible value and type_ will get a TypeError.

toString()

property type_: for property getter

property value: for property getter

class fdi.dataset.metadata.StringParameter(**kwds)

Bases: Parameter

has a unicode string as the value.

serializable(): Can be encoded with serializableEncoder

fdi.dataset.metadata.logger = <Logger fdi.dataset.metadata (WARNING)>: Allowed Parameter types and the corresponding classes. The keys are mnemonics for humans; the values are type(x).__name__.

fdi.dataset.metadataholder module

class fdi.dataset.metadataholder.MetaDataHolder(**kwds)

Bases: object

Object holding meta data. mh: object for compatibility with python2

getMeta(): Returns the current MetaData container of this object. Cannot become a python property because setMeta is in Attributable

hasMeta(): whether the metadata holder is present. During initialization subclass of MetaDataHolder may need to know if the metadata holder has been put in place with is method.

fdi.dataset.ndprint module

fdi.dataset.ndprint.ndprint(data, trans=True, table=True): makes a formated string of an N-dimensional array for printing. The fastest changing index is the innerest list. E.g. A 2 by 3 matrix is [[1,2],[3,4],[5,6]] written as 1 2 3 4 5 6 But if the matrix is a table, the cells in a column change the fastest, and the columns are written vertically. So to print a matrix as a table, whose columns are the innerest list, set trans = True (default) then the matrix needs to be printed transposed: 1 3 5 2 4 6

fdi.dataset.odict module

class fdi.dataset.odict.ODict

Bases: OrderedDict

OrderedDict with a better __repre__.

toString(matprint=None, trans=True)

fdi.dataset.odict.bstr(x, tostr=True, quote="'", **kwds): returns the best string representation. if the object is a string, return single-quoted; if has toString(), use it; else returns str().

fdi.dataset.product module

class fdi.dataset.product.Product(description='UNKOWN', type_='Product', creator='UNKOWN', creationDate=2017-01-01T00:00:00.000000 TAI(0), rootCause='UNKOWN', schema='0.4', startDate=2017-01-01T00:00:00.000000 TAI(0), endDate=2017-01-01T00:00:00.000000 TAI(0), instrument='UNKOWN', modelName='UNKOWN', mission='_AGS', **kwds)

Bases: BaseProduct

Product class (level ALL) version 0.5 inheriting BaseProduct. Automatically generated from fdi/dataset/resources/Product.yml on 2020-06-19 12:49:05.502009.

Generally a Product (inheriting BaseProduct) has project-wide attributes and can be extended to define a plethora of specialized products.

productInfo = {'metadata': {'creationDate': {'data_type': 'finetime', 'default': '0', 'description': 'Creation date of this product', 'fits_keyword': 'DATE', 'unit': 'None'}, 'creator': {'data_type': 'string', 'default': 'UNKOWN', 'description': 'Generator of this product. Example name of institute, organization, person, software, special algorithm etc.', 'fits_keyword': 'CREATOR', 'unit': 'None'}, 'description': {'data_type': 'string', 'default': 'UNKOWN', 'description': 'Description of this product', 'fits_keyword': 'DESCRIPT', 'unit': 'None'}, 'endDate': {'data_type': 'finetime', 'default': '0', 'description': 'Nominal end time of this product.', 'fits_keyword': 'DATE_END', 'unit': 'None', 'valid': ''}, 'instrument': {'data_type': 'string', 'default': 'UNKOWN', 'description': 'Instrument that generated data of this product', 'fits_keyword': 'INSTRUME', 'unit': 'None', 'valid': ''}, 'mission': {'data_type': 'string', 'default': '_AGS', 'description': 'Name of the mission.', 'fits_keyword': 'TELESCOP', 'unit': 'None', 'valid': ''}, 'modelName': {'data_type': 'string', 'default': 'UNKOWN', 'description': 'Model name of the instrument of this product', 'fits_keyword': 'MODEL', 'unit': 'None', 'valid': ''}, 'rootCause': {'data_type': 'string', 'default': 'UNKOWN', 'description': 'Reason of this run of pipeline.', 'fits_keyword': 'ROOTCAUS', 'unit': 'None'}, 'schema': {'data_type': 'string', 'default': '0.4', 'description': 'Version of product schema', 'fits_keyword': 'SCHEMA', 'unit': 'None', 'valid': ''}, 'startDate': {'data_type': 'finetime', 'default': '0', 'description': 'Nominal start time of this product.', 'fits_keyword': 'DATE_OBS', 'unit': 'None', 'valid': ''}, 'type': {'data_type': 'string', 'default': 'Product', 'description': 'Product Type identification. Fully qualified Python class name or CARD.', 'fits_keyword': 'TYPE', 'unit': 'None', 'valid': ''}}}

fdi.dataset.quantifiable module

class fdi.dataset.quantifiable.Quantifiable(unit=None, **kwds)

Bases: object

A Quantifiable object is a numeric object that has a unit. $ x.unit = ELECTRON_VOLTS $ print x.unit eV [1.60218E-19 J]

getUnit(): Returns the unit related to this object.

setUnit(unit): Sets the unit of this object.

property unit

fdi.dataset.serializable module

class fdi.dataset.serializable.Serializable(**kwds)

Bases: object

mh: Can be serialized. Has a ClassID and version instance property to show its class and version information.

serializable(): returns an odict that has all state info of this object. Subclasses should override this function.

serialized(indent=None)

class fdi.dataset.serializable.SerializableEncoder(*, skipkeys=False, ensure_ascii=True, check_circular=True, allow_nan=True, sort_keys=False, indent=None, separators=None, default=None)

Bases: JSONEncoder

can encode parameter and product etc such that they can be recovered with deserializeClassID. Python 3 treats string and unicode as unicode, encoded with utf-8, byte blocks as bytes, encoded with utf-8. Python 2 treats string as str and unicode as unicode, encoded with utf-8, byte blocks as str, encoded with utf-8

default(obj)

Implement this method in a subclass such that it returns a serializable object for o, or calls the base implementation (to raise a TypeError).

For example, to support arbitrary iterators, you could implement default like this:

def default(self, o):
    try:
        iterable = iter(o)
    except TypeError:
        pass
    else:
        return list(iterable)
    # Let the base class default method raise the TypeError
    return JSONEncoder.default(self, o)

fdi.dataset.serializable.serializeClassID(o, indent=None): return JSON using special encoder SerializableEncoder

fdi.dataset.serializable.serializeHipe(o): return JSON using special encoder SerializableHipeEncoder

fdi.dataset.yaml2python module

fdi.dataset.yaml2python.mkinfo(attrs, indent, demo, onlyInclude): make productInfo string from attributes given.

fdi.dataset package

Submodules

fdi.dataset.abstractcomposite module

fdi.dataset.annotatable module

fdi.dataset.attributable module

fdi.dataset.baseproduct module

fdi.dataset.classes module

fdi.dataset.collectionsMockUp module

fdi.dataset.composite module

fdi.dataset.copyable module

fdi.dataset.dataset module

fdi.dataset.datatypes module

fdi.dataset.datawrapper module

fdi.dataset.deserialize module

fdi.dataset.eq module

fdi.dataset.finetime module

fdi.dataset.listener module

fdi.dataset.metadata module

fdi.dataset.metadataholder module

fdi.dataset.ndprint module

fdi.dataset.odict module

fdi.dataset.product module

fdi.dataset.quantifiable module

fdi.dataset.serializable module

fdi.dataset.yaml2python module

fdi

Navigation

Related Topics