fdi.dataset 包
子包
子模块
fdi.dataset.abstractcomposite 模块
- class fdi.dataset.abstractcomposite.AbstractComposite(**kwds)[源代码]
基类:
fdi.dataset.attributable.Attributable
,fdi.dataset.listener.EventListener
,fdi.dataset.composite.Composite
,fdi.dataset.datawrapper.DataWrapperMapper
Composite 的一个可注释和可归属的子类。
Composite 通过 DataContainer 通过 EventListener 继承 annotatable。
fdi.dataset.annotatable 模块
fdi.dataset.arraydataset 模块
- class fdi.dataset.arraydataset.ArrayDataset(data=None, unit=None, description=None, typ_=None, typecode=None, version=None, zInfo=None, alwaysMeta=True, **kwds)[源代码]
基类:
fdi.dataset.dataset.GenericDataset
,collections.abc.Iterable
,fdi.dataset.shaped.Shaped
包含单个数组数据对象的特殊数据集。
mh:如果在实例化过程中省略参数名称,例如 ArrayDataset(a, b, c),假设顺序是数据,单位,描述。 mh:包含提供方法 count()、index()、remove()、reverse() 的序列。 可变序列还需要 append()、extend()、insert()、pop() 和 sort()。
- 数据
此数据集的有效载荷数据。 默认为无,返回 ——-
初始化一个 ArrayDataset。
默认
`None`
会将元数据参数初始化为它们的默认值。 如果data
不是 None 并且具有形状(len
适用),shape
MDP 设置为实际的data
形状。- __init__(data=None, unit=None, description=None, typ_=None, typecode=None, version=None, zInfo=None, alwaysMeta=True, **kwds)[源代码]
初始化一个 ArrayDataset。
默认
`None`
会将元数据参数初始化为它们的默认值。 如果data
不是 None 并且具有形状(len
适用),shape
MDP 设置为实际的data
形状。
- string(level=0, tablefmt='grid', tablefmt1='simple', tablefmt2='simple', width=0, param_widths=None, matprint=None, trans=True, center=- 1, heavy=True, **kwds)
matprint:外部矩阵打印功能 trans:打印转置的二维矩阵。 默认为真。
- class fdi.dataset.arraydataset.Column(*args, typ_='Column', **kwds)[源代码]
基类:
fdi.dataset.arraydataset.ArrayDataset
,fdi.dataset.listener.ColumnListener
列是表格的垂直切割,所有单元格都具有相同的签名。
Column 包含原始 ArrayData,以及可选的描述和单位。 例子::
table = TableDataset() table.addColumn("Energy",Column(data=[1,2,3,4],description="desc",unit='eV'))
初始化一个 ArrayDataset。
默认
`None`
会将元数据参数初始化为它们的默认值。 如果data
不是 None 并且具有形状(len
适用),shape
MDP 设置为实际的data
形状。
fdi.dataset.arraydataset_datamodel 模块
fdi.dataset.attributable 模块
- class fdi.dataset.attributable.Attributable(meta=None, zInfo=None, alwaysMeta=False, **kwds)[源代码]
基类:
fdi.dataset.metadataholder.MetaDataHolder
Attributable 对象是具有元数据概念的对象。
MetaData Porperties (MDPs) are Attributes that store their properties in te metadata table.
Pick out arguments listed in zInfo then put updated parameters into MetaData meta.
meta: meta data container. zInfo: configuration alwaysMeta: always treat parameters as MetaDataProperties (MDPs).
- __init__(meta=None, zInfo=None, alwaysMeta=False, **kwds)[源代码]
Pick out arguments listed in zInfo then put updated parameters into MetaData meta.
meta: meta data container. zInfo: configuration alwaysMeta: always treat parameters as MetaDataProperties (MDPs).
- makeMdpMethods(outputfile=None)[源代码]
Generates a mix-in class file according to MetaData Property list.
- property meta
- fdi.dataset.attributable.MdpInfo = {}
Names not for mormal properties.
- class fdi.dataset.attributable.MetaDataProperties[源代码]
基类:
object
Mix-in place-holder for Class denifitions that has not their own property.py generated by :meth: makeMdpMethods.
- fdi.dataset.attributable.Reserved_Property_Names = ['history', 'meta', 'refs', 'dataset', 'zInfo', '_MDP', 'extraMdp', 'alwaysMeta', 'toString', 'string', 'yaml', 'tree']
These MetaData Parameters (MDPs) and vital attrbutes are Set By Parent classes: | Special MDPs and attrbutes | set by parent Classes | attribute holder | | ‘meta’ | Attributable | _meta | | ‘description’ | Annotatable | description | | ‘data’ and ‘shape’ | DataWrapper | _data | | ‘listeners’ | EventSender | _listener |
- fdi.dataset.attributable.addMetaDataProperty(cls)[源代码]
mh: Add MDP to a class so that although they are metadata, they can be accessed by for example, productfoo.creator.
dynamic properties see https://stackoverflow.com/a/2584050 https://stackoverflow.com/a/1355444
- fdi.dataset.attributable.make_class_properties(attrs, reserved=None)[源代码]
Generates class properties source code string according to given attribute names.
- fdi.dataset.attributable.value2parameter(name, value, descriptor)[源代码]
returns a parameter with correct type and attributes according to its value and name.
value: type must be compatible with data_type. For example [0, 0] is wrong; Vector2d([0, 0)] is right if
data_type``==``vector2d
. descriptor: is zInfo(‘metadata’] or zInfo[‘dataset’][xxx]
fdi.dataset.baseproduct module
- class fdi.dataset.baseproduct.BaseProduct(description='UNKNOWN', typ_='BaseProduct', level='ALL', creator='UNKNOWN', creationDate=FineTime(Unknown), rootCause='UNKNOWN', version='0.8', FORMATV='1.6.0.10', zInfo=None, **kwds)[源代码]
基类:
fdi.dataset.abstractcomposite.AbstractComposite
,fdi.dataset.copyable.Copyable
,fdi.dataset.listener.EventSender
A BaseProduct is the starting point of te whole product tree, and a generic result that can be passed on between processes.
In general a Product contains zero or more datasets, history, optional reference pointers and metadata some required metadata fields. A Product is expected to fully describe itself; this includes the way how this product was achieved (its history). As it is the result of a process, it should be able to save to and restore from an Archive device.
Many times a Product may contain a single dataset and for this purpose the first dataset entry can be accessed by the getDefault() method. Note that the datasets may be a composite of datasets by themselves.
A built-in attributes in Model[‘metadata’] (“MetaData Parameter” or MDP) can be accessed with e.g.
p.creator
, or p.meta[‘creator’].value:p.creator='foo' assert p.creatur=='foo' assert p.meta['creator']=='foo' p.meta['creator']=Parameter('bar') assert p.meta['creator']==Parameter('bar')
BaseProduct class schema 1.6 inheriting [None].
Automatically generated from fdi/dataset/resources/BaseProduct.yml on 2021-12-13 09:38:07.494042.
Description: FDI base class data model
- property FORMATV
- property creationDate
- property creator
- property description
- property history
xx 必须是
self.xx = yy
的属性,才能在 xx 也被子类设置为属性后在超类中工作。
- property level
- property rootCause
- string(level=0, tablefmt='grid', tablefmt1='rst', tablefmt2='rst', matprint=None, trans=True, beforedata='', **kwds)
like AbstractComposite but with history
- targetChanged(event)[源代码]
Informs that an event has happened in a target of any type. Paremeters ———-
- toString(level=0, tablefmt='grid', tablefmt1='rst', tablefmt2='rst', matprint=None, trans=True, beforedata='', **kwds)[源代码]
like AbstractComposite but with history
- property type
- property version
fdi.dataset.booleanparameter module
fdi.dataset.browseproduct module
- class fdi.dataset.browseproduct.BrowseProduct(description='UNKNOWN', typ_='BrowseProduct', level='ALL', creator='UNKNOWN', creationDate=FineTime(Unknown), rootCause='UNKNOWN', version='0.8', FORMATV='1.6.0.1', zInfo=None, **kwds)[源代码]
基类:
fdi.dataset.baseproduct.BaseProduct
BrowseProduct class schema 1.6 inheriting [‘BaseProduct’].
Automatically generated from fdi/dataset/resources/BrowseProduct.yml on 2021-12-13 09:38:07.543630.
Description: Container of media data for browsing.
Generally a Product (inheriting BaseProduct) has project-wide attributes and can be extended to define a plethora of specialized products.
Initializes instances with more metadata as attributes, set to default values.
Put description keyword argument here to allow e.g. BaseProduct(“foo”) and description=’foo’
- __init__(description='UNKNOWN', typ_='BrowseProduct', level='ALL', creator='UNKNOWN', creationDate=FineTime(Unknown), rootCause='UNKNOWN', version='0.8', FORMATV='1.6.0.1', zInfo=None, **kwds)[源代码]
Initializes instances with more metadata as attributes, set to default values.
Put description keyword argument here to allow e.g. BaseProduct(“foo”) and description=’foo’
fdi.dataset.classes module
- class fdi.dataset.classes.Classes[源代码]
基类:
object
A dictionary of class names and their class objects that are allowed to be deserialized.
An fdi package built-in dictionary (in the format of locals() output) is kept internally. Users who need add more deserializable class can for example:
Define new classes ``class Myclass():
….``
update Classes
Classes.classes.update({'myClasses': MyClass})
and use
new_instance = Classes.mapping['MyClass']
For a new package with many classes:
Import user classes in a python file for example projectclasses.py:
`` Class PC(Classes):
- module_class = {
‘mypackage.mymodule’: [‘MyClass1’, ‘MyClass2’],
} # from another module defining a dict of modulename-Classobj pairs try:
from mypackage.mymodule import pairs
- except (ImportError, ModuleNotFoundError) as e:
logger.info(e)
- else:
module_class.update(pairs)
_package = {} _classes = {}
To use: `` from fdi.dataset.classes import Classes from my.package.projectclasses import PC prjcls = Classes.mapping Classes.updateMapping(PC.updateMapping())
new_instance = prjcls[‘MyClass1’]
- class fdi.dataset.classes.Classes_meta(*args, **kwds)[源代码]
基类:
type
metaclass for ‘classproperty’. # https://stackoverflow.com/a/1800999
Class is initialized with built-in classes by default.
- importModuleClasses(rerun=False, exclude=None, ignore_error=False, verbose=False)[源代码]
The set of deserializable classes in module_class is maintained by hand.
Do nothing if the classes mapping is already made so repeated calls will not cost more time.
rerun: set to True to force re-import. If the module-class list has never been imported, it will be imported regardless rerun. exclude: modules whose names (without ‘.’) are in exclude are not imported.
- property mapping
Returns the dictionary of classes allowed for deserialization, including the fdi built-ins and user added classes.
Will update the classes if the list is empty
- module_class = {'fdi.dataset.arraydataset': ['ArrayDataset', 'Column'], 'fdi.dataset.baseproduct': ['BaseProduct'], 'fdi.dataset.browseproduct': ['BrowseProduct'], 'fdi.dataset.dataset': ['GenericDataset', 'CompositeDataset'], 'fdi.dataset.datatypes': ['Vector', 'Vector2D', 'Quaternion'], 'fdi.dataset.dateparameter': ['DateParameter'], 'fdi.dataset.deserialize': ['deserialize'], 'fdi.dataset.eq': ['DeepEqual'], 'fdi.dataset.finetime': ['FineTime', 'FineTime1', 'utcobj'], 'fdi.dataset.history': ['History'], 'fdi.dataset.listener': ['ListnerSet'], 'fdi.dataset.mediawrapper': ['MediaWrapper'], 'fdi.dataset.metadata': ['AbstractParameter', 'Parameter', 'MetaData'], 'fdi.dataset.numericparameter': ['NumericParameter', 'BooleanParameter'], 'fdi.dataset.odict': ['ODict'], 'fdi.dataset.product': ['Product'], 'fdi.dataset.readonlydict': ['ReadOnlyDict'], 'fdi.dataset.serializable': ['Serializable'], 'fdi.dataset.stringparameter': ['StringParameter'], 'fdi.dataset.tabledataset': ['TableDataset', 'IndexedTableDataset'], 'fdi.dataset.testproducts': ['TP', 'TC', 'TM'], 'fdi.dataset.unstructureddataset': ['UnstrcturedDataset'], 'fdi.pal.context': ['AbstractContext', 'Context', 'MapContext', 'RefContainer', 'ContextRuleException'], 'fdi.pal.productref': ['ProductRef'], 'fdi.pal.query': ['AbstractQuery', 'MetaQuery', 'StorageQuery'], 'fdi.pal.urn': ['Urn']}
- fdi.dataset.classes.logger = <Logger fdi.dataset.classes (WARNING)>
this has to be in a different file where other interface classes are defined to avoid circular dependency (such as , Serializable.
- Type
Note
fdi.dataset.collectionsMockUp module
fdi.dataset.composite module
- class fdi.dataset.composite.Composite(data=None, **kwds)[源代码]
基类:
fdi.dataset.datawrapper.DataContainer
,fdi.dataset.serializable.Serializable
,collections.abc.MutableMapping
A container of named Datasets.
This container can hold zero or more datasets, each of them stored against a unique name. The order of adding datasets to this Composite is important, that is: the keySet() method will return a set of labels of the datasets in the sequence as they were added. Note that replacing a dataset with the same name, will keep the order.
DeepEqual
must stay to the left ofMutableMapping
so its __eq__ will get to run. Serializable becomes a parent for having __setstate__.- 数据
default None will init with a dict.
- get(name, default=Invalid)[源代码]
Returns the dataset to which this composite maps the specified name.
If the attrbute does not exist and default unspecified, raise a KeyError.
- 参数
default: assigne to None or anything for missing name.
- getDatasetNames()
Returns a list view of the keys contained in this composite.
- remove(name)[源代码]
Removes the mapping for this name from this composite. mh: returns None if name is None or item does not exist.
- set(name, dataset)[源代码]
Associates the specified dataset with the specified key in this map(optional operation). If the map previously contained a mapping for this key, the old dataset is replaced by the specified dataset. this composite does not permit null or empty keys.
- string(level=0, matprint=None, trans=True, **kwds)
- class fdi.dataset.composite.UserDictAdapter(data=None, *args, **kwds)[源代码]
基类:
collections.UserDict
Adapter class to make UserDict cooperative to multiple inheritance and take data keyword arg.
REf. https://rhettinger.wordpress.com/2011/05/26/super-considered-super/
- 参数
data: initialize UserDict.
fdi.dataset.copyable module
fdi.dataset.dataset module
- class fdi.dataset.dataset.CompositeDataset(**kwds)[源代码]
基类:
fdi.dataset.listener.MetaDataListener
,fdi.dataset.abstractcomposite.AbstractComposite
An CompositeDataset is a Dataset that contains zero or more named Datasets. It allows to build arbitrary complex dataset structures.
It also defines the iteration ordering of its children, which is the order in which the children were inserted into this dataset.
- class fdi.dataset.dataset.Dataset(**kwds)[源代码]
基类:
fdi.dataset.attributable.Attributable
,fdi.dataset.datawrapper.DataContainer
,fdi.dataset.serializable.Serializable
,fdi.dataset.listener.MetaDataListener
Attributable and annotatable information data container that can be be part of a Product.
developer notes: The intent is that developers do not derive from this interface
directly. Instead, they should inherit from one of the generic datasets that this package provides:
mh: GenericDataset, UnstrcturedDataset ArrayDataset. TableDataset or CompositeDataset.
- accept(visitor)[源代码]
Hook for adding functionality to object through visitor pattern. Parameter ———
- string(level=0, tablefmt='grid', tablefmt1='simple', tablefmt2='rst', param_widths=None, width=0, matprint=None, trans=True, heavy=True, center=- 1, **kwds)
matprint: an external matrix print function trans: print 2D matrix transposed. default is True. Parameter ———
- class fdi.dataset.dataset.GenericDataset(**kwds)[源代码]
基类:
fdi.dataset.dataset.Dataset
,fdi.dataset.typed.Typed
,fdi.dataset.datawrapper.DataWrapper
mh: Contains one typed data item with a unit and a typecode.
fdi.dataset.datatypes module
- fdi.dataset.datatypes.DataTypes = {'': 'None', 'baseProduct': 'BaseProduct', 'binary': 'int', 'boolean': 'bool', 'byte': 'int', 'finetime': 'FineTime', 'finetime1': 'FineTime1', 'float': 'float', 'hex': 'int', 'integer': 'int', 'list': 'list', 'mapContext': 'MapContext', 'product': 'Product', 'quaternion': 'Quaternion', 'short': 'int', 'string': 'str', 'tuple': 'tuple', 'vector': 'Vector', 'vector2d': 'Vector2D'}
maps class type names to human-friendly types
- class fdi.dataset.datatypes.Quaternion(components=None, **kwds)[源代码]
基类:
fdi.dataset.datatypes.Vector
Quaternion with 4-component data.
invoked with no argument results in a vector of [0, 0, 0, 0] components
- class fdi.dataset.datatypes.Vector(components=None, **kwds)[源代码]
基类:
fdi.dataset.quantifiable.Quantifiable
,fdi.dataset.serializable.Serializable
,fdi.dataset.eq.DeepcmpEqual
N dimensional vector.
If unit, description, type etc meta data is needed, use a Parameter.
A Vector can compare with a value whose type is in
DataTypes
, the quantity being used is the magnitude.invoked with no argument results in a vector of [0, 0, 0] components.
- __init__(components=None, **kwds)[源代码]
invoked with no argument results in a vector of [0, 0, 0] components.
- property components
for property getter
- getComponents()[源代码]
Returns the actual components that is allowed for the components of this vector.
- string(level=0, **kwds)
- class fdi.dataset.datatypes.Vector2D(components=None, **kwds)[源代码]
基类:
fdi.dataset.datatypes.Vector
Vector with 2-component data
invoked with no argument results in a vector of [0, 0] components
- fdi.dataset.datatypes.cast(val, typ_, namespace=None)[源代码]
casts the input value to type specified, which is in DataTypeNames.
For example ‘binary’ type ‘0x9’ is casted to int 9.
namespace: default is Classes.mapping.
- fdi.dataset.datatypes.logger = <Logger fdi.dataset.datatypes (WARNING)>
Allowed data (Parameter and Dataset) types and the corresponding classe names. The keys are mnemonics for humans; the values are type(x).__name__.
fdi.dataset.datawrapper module
- class fdi.dataset.datawrapper.DataContainer(data=None, **kwds)[源代码]
基类:
fdi.dataset.annotatable.Annotatable
,fdi.dataset.copyable.Copyable
,fdi.dataset.eq.DeepcmpEqual
,collections.abc.Container
A DataContainer is a composite of data and description.
mh: note that There is no metadata. Implemented partly from AbstractDataWrapper.
data: a Container. Default is None.
- property data
- getData()[源代码]
Returns the data in this dw
- 返回
- 返回类型
self._data or None if ‘._data’ is missing. e.g. subclass overriding setData calling getData from its setData that is called by ‘this.__init__’.
- class fdi.dataset.datawrapper.DataWrapper(*args, **kwds)[源代码]
基类:
fdi.dataset.datawrapper.DataContainer
,fdi.dataset.quantifiable.Quantifiable
A DataWrapper is a composite of data, unit and description.
mh: note that all data are in the same unit. There is no metadata. Implemented from AbstractDataWrapper.
fdi.dataset.dateparameter module
- class fdi.dataset.dateparameter.DateParameter(value=None, description='UNKNOWN', default=None, valid=None, **kwds)[源代码]
基类:
fdi.dataset.metadata.Parameter
,fdi.dataset.typecoded.Typecoded
has a FineTime as the value.
Set up a parameter whose value is a point in TAI time.
- class fdi.dataset.dateparameter.DateParameter1(value=None, description='UNKNOWN', default=None, valid=None, **kwds)[源代码]
基类:
fdi.dataset.dateparameter.DateParameter
Like DateParameter but usese FineTime1.
Set up a parameter whose value is a point in TAI time.
fdi.dataset.deserialize module
- class fdi.dataset.deserialize.IntDecoder(*, object_hook=None, parse_float=None, parse_int=None, parse_constant=None, strict=True, object_pairs_hook=None)[源代码]
基类:
json.decoder.JSONDecoder
adapted from https://stackoverflow.com/questions/45068797/how-to-convert-string-int-json-into-real-int-with-json-loads modified to also convert keys in dictionaries.
object_hook
, if specified, will be called with the result of every JSON object decoded and its return value will be used in place of the givendict
. This can be used to provide custom deserializations (e.g. to support JSON-RPC class hinting).object_pairs_hook
, if specified will be called with the result of every JSON object decoded with an ordered list of pairs. The return value ofobject_pairs_hook
will be used instead of thedict
. This feature can be used to implement custom decoders that rely on the order that the key and value pairs are decoded (for example, collections.OrderedDict will remember the order of insertion). Ifobject_hook
is also defined, theobject_pairs_hook
takes priority.parse_float
, if specified, will be called with the string of every JSON float to be decoded. By default this is equivalent to float(num_str). This can be used to use another datatype or parser for JSON floats (e.g. decimal.Decimal).parse_int
, if specified, will be called with the string of every JSON int to be decoded. By default this is equivalent to int(num_str). This can be used to use another datatype or parser for JSON integers (e.g. float).parse_constant
, if specified, will be called with one of the following strings: -Infinity, Infinity, NaN. This can be used to raise an exception if invalid JSON numbers are encountered.If
strict
is false (true is the default), then control characters will be allowed inside strings. Control characters in this context are those with character codes in the 0-31 range, including'\t'
(tab),'\n'
,'\r'
and'\0'
.
- class fdi.dataset.deserialize.IntDecoderOD(*, object_hook=None, parse_float=None, parse_int=None, parse_constant=None, strict=True, object_pairs_hook=None)[源代码]
基类:
fdi.dataset.deserialize.IntDecoder
object_hook
, if specified, will be called with the result of every JSON object decoded and its return value will be used in place of the givendict
. This can be used to provide custom deserializations (e.g. to support JSON-RPC class hinting).object_pairs_hook
, if specified will be called with the result of every JSON object decoded with an ordered list of pairs. The return value ofobject_pairs_hook
will be used instead of thedict
. This feature can be used to implement custom decoders that rely on the order that the key and value pairs are decoded (for example, collections.OrderedDict will remember the order of insertion). Ifobject_hook
is also defined, theobject_pairs_hook
takes priority.parse_float
, if specified, will be called with the string of every JSON float to be decoded. By default this is equivalent to float(num_str). This can be used to use another datatype or parser for JSON floats (e.g. decimal.Decimal).parse_int
, if specified, will be called with the string of every JSON int to be decoded. By default this is equivalent to int(num_str). This can be used to use another datatype or parser for JSON integers (e.g. float).parse_constant
, if specified, will be called with one of the following strings: -Infinity, Infinity, NaN. This can be used to raise an exception if invalid JSON numbers are encountered.If
strict
is false (true is the default), then control characters will be allowed inside strings. Control characters in this context are those with character codes in the 0-31 range, including'\t'
(tab),'\n'
,'\r'
and'\0'
.
- fdi.dataset.deserialize.constructSerializable(obj, lookup=None, debug=False)[源代码]
mh: reconstruct object from the output of jason.loads(). Recursively goes into nested class instances that are not encoded by default by JSONEncoder, instantiate and fill in variables. Objects to be deserialized must have their classes loaded. _STID cannot have module names in it (e.g. dataset.Product) or locals()[classname] or globals()[classname] will not work.
- fdi.dataset.deserialize.deserialize(js, lookup=None, debug=False, usedict=True)[源代码]
Loads classes with _STID from the results of serialize.
if usedict is True dict insted of ODict will be used.
- fdi.dataset.deserialize.deserialize_args(all_args, not_quoted=False, first_string=True, serialize_out=False)[源代码]
parse the command path to get positional and keywords arguments.
if not_quoted is True, split everythine to the left of first { with Serialize_Args_Sep append the part startin from the {. mark=’{’
else after splitting all_args with Serialize_Args_Sep: mark=’%7B%22’ (quote(‘{‘))
Scan from left. if all_args[i] not start with mark
Conversion rules: |all_args[i]| converted to | | else | convert (case insensitive) and move on to the next segment | | string not starting with
`'0x'`
| quote |else decode_str() if
`not_quoted==False`
else only substitute SAS_Avatar with Serialize_Args_Sep. Then deserialize() this segment to become`{'apiargs':list, 'foo':bar ...}`
, append value of`apiargs`
to the converted-list above, remove the`apiargs`
-`val`
pair.return 200 as the reurn code followed by the converted-list and the deserialized
`dict`
.
- All_args
a list of path segments for the args list.
- First_string
Do not try to change the type of the first arg (assumed to be the function/method name).
- fdi.dataset.deserialize.encode_str(a0)[源代码]
quote to remove general url offenders then use a mostly harmless str to substitute Serialize_Args_Sep.
- fdi.dataset.deserialize.logger = <Logger fdi.dataset.deserialize (WARNING)>
this has to be in a different file where other interface classes are defined to avoid circular dependency (such as , Serializable.
- Type
Note
- fdi.dataset.deserialize.serialize_args(*args, not_quoted=False, **kwds)[源代码]
Serialize all positional and keywords arguements as they would appear in a function call. Arguements are assumed to have been placed in the same order of a valid function/method call. They are scanned from left to right from args[i] i = 0, 1,… to kwds[j] j = 0, 1, …
Scan args from i=0. if is of args[i] is of bool, int, float types, convert with str, if str(), convert with encode_str(), if bytes or bytearray’ types, with ```0x```+`hex(), save to the convered-list, and move on to the next element.
else if finding a segment not of any of the above types,
** put this and the rest of
`args`
as the`value`
in`{'apiargs':value}`
, ** and append kwds key-val pairs after this pair, ** serialize the disctionary with serialize() and encode_str() ** append the result to the converted-list. ** break from the args scan loop. * if args scan loop reaches its end, if kwds is not empty, serialize it with serialize() and encode_str(), or scanning reaches the end of args. * append the result to the converted-list. * join the converted-list with Serialize_Args_Sep. * return the result string
fdi.dataset.eq module
- fdi.dataset.eq.DeepEqual
alias of
fdi.dataset.eq.DeepcmpEqual
- class fdi.dataset.eq.DeepcmpEqual(*args, **kwds)[源代码]
基类:
object
Equality tested by deepcmp.
Must pass args so DataWrapper in Composite can get data.
- class fdi.dataset.eq.EqualDict(**kwds)[源代码]
基类:
object
mh: Can compare key-val pairs of another object with self. False if compare with None or exceptions raised, e.g. obj does not have items()
- class fdi.dataset.eq.EqualODict(**kwds)[源代码]
基类:
object
mh: Can compare order and key-val pairs of another object with self. False if compare with None or exceptions raised, e.g. obj does not have items()
- class fdi.dataset.eq.StateEqual(*args, **kwds)[源代码]
基类:
object
Equality tested by hashed state.
Must pass args so DataWrapper in Composite can get data.
- equals(obj, **kwds)
compares hash.
- fdi.dataset.eq.deepcmp(obj1, obj2, seenlist=None, verbose=False, eqcmp=False)[源代码]
Recursively descends into obj1’s every component and compares with its counterpart in obj2.
Factors includes; * if they are the same object * type * quick string *
`__eq__`
or`__cmp__`
if requested * state from`__getstate__`
* quick length * members if is`Mapping`
,`Sequence`
( set, list, dict, ordereddict, UserDict … ) * properties/attributes in`__dict__`
Detects cyclic references.
- 返回
None
if finds no difference, a string of explanationotherwise.
eqcmp: if True, use __eq__ or __cmp__ if the objs have them. If False only use as the last resort. default True.
fdi.dataset.finetime module
- class fdi.dataset.finetime.FineTime(date=None, format=None, **kwds)[源代码]
基类:
fdi.dataset.copyable.Copyable
,fdi.dataset.eq.DeepcmpEqual
,fdi.dataset.serializable.Serializable
Atomic time (SI seconds) elapsed since the TAI epoch of 1 January 1958 UT2. The resolution is one microsecond and the allowable range is: epoch + /-290, 000 years approximately.
This has the following advantages, compared with the standard class:
It has better resolution(microseconds) Time differences are correct across leap seconds It is immutable.
Initiate with a UTC date or an integer TAI.
date; time to be set to. Acceptable types: int for TAI, datetime.datetime, string for ISO format date-time, or bytes-like classed that can get string by calling its decode(encoding=’utf-8’)
- DEFAULT_FORMAT = '%Y-%m-%dT%H:%M:%S.%f'
- EPOCH = datetime.datetime(1958, 1, 1, 0, 0, tzinfo=datetime.timezone.utc)
number of TAI units in a second
- RESOLUTION = 1000000
- RETURNFMT = '%s.%06d'
- TIMESPEC = 'microseconds'
- UTC_LOW_LIMIT = datetime.datetime(1958, 1, 1, 0, 0, tzinfo=datetime.timezone.utc)
- __init__(date=None, format=None, **kwds)[源代码]
Initiate with a UTC date or an integer TAI.
date; time to be set to. Acceptable types: int for TAI, datetime.datetime, string for ISO format date-time, or bytes-like classed that can get string by calling its decode(encoding=’utf-8’)
- classmethod datetimeToFineTime(dtm)[源代码]
Return given Python Datetime in FineTime to the precision of the input. Rounded to the last digit. Unit is decided by RESOLUTION.
- getFormat()[源代码]
format cannot be a property name as it is a built so`@format.setter` is not allowed.
- isoutc(format='%Y-%m-%dT%H:%M:%S.%f')[源代码]
Returns a String representation of this objet in ISO format without timezone. sub-second set to TIMESPEC.
If tai is None return ‘’. ;format: time format. default ‘%Y-%m-%dT%H:%M:%S’ prints like 2019-02-17T12:43:04.577000
- microsecondsSinceEPOCH()[源代码]
Return the rounded integer number of microseconds since the epoch: 1 Jan 1958.
- setTime(time)[源代码]
Sets the time of this object.
If an integer is given, it will be taken as the TAI. If a datetime object or a string code is given, the timezone will be set to UTC. Only when current TAI is 0, so a non-zero instance is immutable. Violation gets a TypeError.
- string(level=0, width=0, **kwds)
Returns a String representation of this object according to self.format. level: 0 prints like 2019-02-17T12:43:04.577000 TAI(…) width: if non-zero, insert newline to break simplified output into shorter lines. For level=0 it is
` #TODO `
- property time
- toDate(tai=None)
Return given FineTime as a Python Datetime.
tai: if not given or given as None, return this object’s time as a Python Datetime.
- class fdi.dataset.finetime.FineTime1(*args, **kwds)[源代码]
基类:
fdi.dataset.finetime.FineTime
Same as FineTime but Epoch is 2017-1-1 0 UTC and unit is millisecond
Initiate with a UTC date or an integer TAI.
date; time to be set to. Acceptable types: int for TAI, datetime.datetime, string for ISO format date-time, or bytes-like classed that can get string by calling its decode(encoding=’utf-8’)
- EPOCH = datetime.datetime(2017, 1, 1, 0, 0, tzinfo=datetime.timezone.utc)
number of TAI units in a second
- RESOLUTION = 1000
- RETURNFMT = '%s.%03d'
- TAI_AT_EPOCH = 1861920037000000
- TIMESPEC = 'milliseconds'
fdi.dataset.history module
- class fdi.dataset.history.History(other=None, **kwds)[源代码]
基类:
fdi.dataset.dataset.CompositeDataset
Public interface to the history dataset. Contains the main methods for retrieving a script and copying the history.
mh: The copy constructor is better not be implemented. Use copy() instead. Remember: not only copies the datasets, but also changes the history ID in the metadata and relevant table entries to indicate that this a new independent product of which the history may change.
- __init__(other=None, **kwds)[源代码]
mh: The copy constructor is better not be implemented. Use copy() instead. Remember: not only copies the datasets, but also changes the history ID in the metadata and relevant table entries to indicate that this a new independent product of which the history may change.
fdi.dataset.indexed module
- class fdi.dataset.indexed.Indexed(indexPattern=None, **kwds)[源代码]
基类:
object
Abstract class with an index table for efficient records look-up.
indexPattern: specifies which columns to use to do look up.
- __init__(indexPattern=None, **kwds)[源代码]
indexPattern: specifies which columns to use to do look up.
- getColumnsToLookup()[源代码]
returns an iterator that gives a number of sequences to looking up over.
Default is a tuple of the
data
attributes of every columns specified byindexPattern
. To be overrided for different data model.
- property indexPattern
- property toc
returns the index table of content.
- updateToc(which=None, for_records=None)[源代码]
Build index in format specified in indexPattern for retrieving record. which: an iterator that gives a number of sequences to looking up over. Default is
getColumnsToLookup()
. for_records: a list or aSlice
of record (row) numbers. Those are changed records that caused updating. default is all records.
- vLookUp(key, return_index=True, multiple=False)[源代码]
Similar to Excel VLOOKUP, return all records (rows) that match the key. key: taken as a dictionary key unless
multiple
is True. return_index: if True (default) return index in the array of columns. multiple: if True (default is False) loop through key as a sequence of keys and return a sequece.
fdi.dataset.invalid module
fdi.dataset.listener module
- class fdi.dataset.listener.ColumnListener(**kwds)[源代码]
基类:
fdi.dataset.listener.EventListener
Listener for events occuring in a Column.
Available types:
* DESCRIPTION_CHANGED * UNIT_CHANGED * DATA_CHANGED
Cause is always null.
- class fdi.dataset.listener.DatasetEvent(source, target, typ_, change, cause, rootCause, **kwds)[源代码]
基类:
fdi.dataset.serializable.Serializable
- string(level=0, **kwds)
- class fdi.dataset.listener.DatasetListener(**kwds)[源代码]
基类:
fdi.dataset.listener.EventListener
Listener for events occuring in MetaData. Available types:
* DESCRIPTION_CHANGED, METADATA_CHANGED (all datasets) * DATA_CHANGED, UNIT_CHANGED (ArrayDataset) * COLUMN_ADDED, COLUMN_REMOVED, COLUMN_CHANGED, ROW_ADDED, VALUE_CHANGED (TableDataset) * DATASET_ADDED, DATASET_REMOVED, DATASET_CHANGED (CompositeDataset)
Possible causes:
* not null (METADATA_CHANGED, COLUMN_CHANGED, DATASET_CHANGED) * null (rest)
Warning: The listener handler must be a class attribute in order to create an object hard reference. See DatasetBaseListener.
- class fdi.dataset.listener.EventListener(*args, **kwds)[源代码]
基类:
fdi.dataset.annotatable.Annotatable
Generic interface for listeners that will listen to anything
- class fdi.dataset.listener.EventSender(**kwds)[源代码]
基类:
object
adapted from Peter Thatcher’s https://stackoverflow.com/questions/1092531/event-system-in-python/1096614#1096614
- 参数
Returns
- addListener(listener, cls=<class 'fdi.dataset.listener.EventListener'>)[源代码]
Adds a listener to this. Paremeters ———-
- property listeners
- 参数
Returns
- class fdi.dataset.listener.ListnerSet(data=None, *args, **kwds)[源代码]
基类:
fdi.dataset.serializable.Serializable
,fdi.dataset.eq.DeepcmpEqual
,collections.UserList
Mutable collection of Listeners of an EvenSender.
- 参数
data: default is None for a list.
- string(level=0, alist=False, **kwds)
- 返回
LIST[TUPLE(OBJ)] or STRXS
A list of member-repre tuples or a string of all depending on alist.
- toString(level=0, alist=False, **kwds)[源代码]
- 返回
LIST[TUPLE(OBJ)] or STRXS
A list of member-repre tuples or a string of all depending on alist.
- property urns
- class fdi.dataset.listener.MetaDataListener(*args, **kwds)[源代码]
基类:
fdi.dataset.listener.EventListener
Listener for events occuring in MetaData. Available types:
* PARAMETER_ADDED * PARAMETER_REMOVED * PARAMETER_CHANGED
Possible causes: not null (for PARAMETER_CHANGED, if parameter internally changed) null (for PARAMETER_CHANGED, when set is called with a previous existing parameter, and rest)
Warning: The listener handler must be a class attribute in order to create an object hard reference. See DatasetBaseListener.
- class fdi.dataset.listener.ParameterListener(**kwds)[源代码]
基类:
fdi.dataset.listener.EventListener
Listener for events occuring in a Parameter. Available types:
* DESCRIPTION_CHANGED * UNIT_CHANGED * VALUE_CHANGED * UNKOWN_ATTRIBUTE_CHANGED
Cause is always null.
Warning: The listener handler must be a class attribute in order to create an object hard reference. See DatasetBaseListener.
- class fdi.dataset.listener.ProductListener(**kwds)[源代码]
基类:
fdi.dataset.listener.EventListener
Listener for events occuring in Product. Available types:
* METADATA_CHANGED * DATASET_ADDED * DATASET_REMOVED * DATASET_CHANGED
Possible causes:
* not null (METADATA_CHANGED, DATASET_CHANGED) * null (METADATA_CHANGED, DATASET_REMOVED, DATASET_CHANGED)
Warning: The listener handler must be a class attribute in order to create an object hard reference. See DatasetBaseListener.
fdi.dataset.mediawrapper module
- class fdi.dataset.mediawrapper.MediaWrapper(data=None, unit=None, description=None, typ_='image/png', source=None, typecode=None, version=None, zInfo=None, alwaysMeta=True, **kwds)[源代码]
基类:
fdi.dataset.arraydataset.ArrayDataset
A MediaWrapper contains raw, usually binary, data in specific format.
Initializes media data wrapped in ArrayDataset.
typ_: www style string that follows Content-Type: `. Default is `imagw/png.
fdi.dataset.mediawrapper_datamodel module
fdi.dataset.messagequeue module
- class fdi.dataset.messagequeue.MqttRelayListener(topics=None, host=None, port=None, username=None, passwd=None, callback=None, clean_session=None, client_id=None, userdata=None, qos=1, **kwds)[源代码]
基类:
fdi.dataset.listener.EventListener
Generic interface for sending anything heard to an MQTT message queue.
Starts a MQTT message queue and forward everything in the arguement list to the MQ serialized.
host, port, username, passwd: if any is not provided, it is looked up in `config[‘mqtt’].
- __init__(topics=None, host=None, port=None, username=None, passwd=None, callback=None, clean_session=None, client_id=None, userdata=None, qos=1, **kwds)[源代码]
Starts a MQTT message queue and forward everything in the arguement list to the MQ serialized.
host, port, username, passwd: if any is not provided, it is looked up in `config[‘mqtt’].
- class fdi.dataset.messagequeue.MqttRelaySender(topics=None, host=None, port=None, username=None, passwd=None, callback=None, clean_session=None, client_id=None, userdata=None, qos=1, **kwds)[源代码]
基类:
fdi.dataset.listener.EventSender
Gets MQTT messages and forwards to listeners.
Starts a MQTT message queue and forward everything in the arguement list to the MQ serialized.
host, port, username, passwd: if any is not provided, it is looked up in `config[‘mqtt’].
- __init__(topics=None, host=None, port=None, username=None, passwd=None, callback=None, clean_session=None, client_id=None, userdata=None, qos=1, **kwds)[源代码]
Starts a MQTT message queue and forward everything in the arguement list to the MQ serialized.
host, port, username, passwd: if any is not provided, it is looked up in `config[‘mqtt’].
fdi.dataset.metadata module
- class fdi.dataset.metadata.AbstractParameter(value=None, description='UNKNOWN', **kwds)[源代码]
基类:
fdi.dataset.annotatable.Annotatable
,fdi.dataset.copyable.Copyable
,fdi.dataset.eq.DeepcmpEqual
,fdi.dataset.listener.DatasetEventSender
,fdi.dataset.serializable.Serializable
Parameter is the interface for all named attributes in the MetaData container.
A Parameter is a variable with associated information about its description, unit, type, valid ranges, default, format code etc. Type can be numeric, string, datetime, vector.
Often a parameter shows a property. So a parameter in the metadata of a dataset or product is often called a property.
Default value=None, description=’UNKNOWN’
Constructed with no argument results in a parameter of None value and ‘UNKNOWN’ description ‘’. With a signle argument: arg -> value, ‘UNKNOWN’ as default-> description. With two positional arguments: arg1-> value, arg2-> description. Type is set according to value’s. Unsuported parameter types will get a NotImplementedError.
- __init__(value=None, description='UNKNOWN', **kwds)[源代码]
Constructed with no argument results in a parameter of None value and ‘UNKNOWN’ description ‘’. With a signle argument: arg -> value, ‘UNKNOWN’ as default-> description. With two positional arguments: arg1-> value, arg2-> description. Type is set according to value’s. Unsuported parameter types will get a NotImplementedError.
- hash()[源代码]
hash and equality derived only from the value of the parameter.
because Python does not allow overriding __eq__ without setting hash to None.
- string(level=0, alist=False, **kwds)
alist: returns a dictionary string representation of parameter attributes.
- toString(level=0, alist=False, **kwds)[源代码]
alist: returns a dictionary string representation of parameter attributes.
- property value
for property getter
- fdi.dataset.metadata.Default_Extra_Param_Width = 10
- Attribute | Defining Module | Holder Variable |‘description’ | Annotatable | description |‘typ_’ | Typed | _type |‘unit’ | Quantifiable | ‘_unit’ |‘typecode’ | Typecoded | ‘_typecode’ |
- class fdi.dataset.metadata.MetaData(copy=None, defaults=None, **kwds)[源代码]
基类:
fdi.dataset.listener.ParameterListener
,fdi.dataset.composite.Composite
,fdi.dataset.copyable.Copyable
,fdi.dataset.listener.DatasetEventSender
A container of named Parameters. A MetaData object can have one or more parameters, each of them stored against a unique name. The order of adding parameters to this container is important, that is: the keySet() method will return a set of labels of the parameters in the sequence as they were added. Note that replacing a parameter with the same name, will keep the order.
- Table_Widths = [{'name': 15, 'value': 18, 'unit': 6, 'type': 8, 'valid': 17, 'default': 15, 'code': 4, 'description': 17}]
- set(name, newParameter)[源代码]
Saves the parameter and adds eventhandling.
In a parameter name, dot or other invalid characters (when the name is used as a property name) is ignored.
Raises TypeError if not given Parameter (sub) class object.
- string(level=0, tablefmt='grid', tablefmt1='simple', tablefmt2='rst', extra=False, param_widths=None, width=0, **kwds)
return string representation of metada.
level: 0 is the most detailed, 2 is the least, tablefmt: format string in packae
tabulate
, for level==0, tablefmt1 for level1, tablefmt2: format of 2D table data. param_widths: controls how the attributes of every parameter are displayed in the table cells. If is set to -1, there is no cell-width limit. For finer control set a dictionary of parameter attitute names and how many characters wide its tsble cell is, 0 for ommiting the attributable. Default is MetaData.Table_Widths[0]. e.g.- ``{‘name’: 8, ‘value’: 17, ‘unit’: 7, ‘type’: 8,
‘valid’: 20, ‘default’: 17, ‘code’: 4, ‘description’: 15}``
- toString(level=0, tablefmt='grid', tablefmt1='simple', tablefmt2='rst', extra=False, param_widths=None, width=0, **kwds)[源代码]
return string representation of metada.
level: 0 is the most detailed, 2 is the least, tablefmt: format string in packae
tabulate
, for level==0, tablefmt1 for level1, tablefmt2: format of 2D table data. param_widths: controls how the attributes of every parameter are displayed in the table cells. If is set to -1, there is no cell-width limit. For finer control set a dictionary of parameter attitute names and how many characters wide its tsble cell is, 0 for ommiting the attributable. Default is MetaData.Table_Widths[0]. e.g.- ``{‘name’: 8, ‘value’: 17, ‘unit’: 7, ‘type’: 8,
‘valid’: 20, ‘default’: 17, ‘code’: 4, ‘description’: 15}``
- class fdi.dataset.metadata.Parameter(value=None, description='UNKNOWN', typ_='', default=None, valid=None, **kwds)[源代码]
基类:
fdi.dataset.metadata.AbstractParameter
,fdi.dataset.typed.Typed
Parameter is the interface for all named attributes in the MetaData container. It can have a value and a description. Default arguments: typ_=’’, default=None, valid=None. value=default, description=’UNKNOWN’
- invoked with no argument results in a parameter of
None value and ‘UNKNOWN’ description ‘’. typ_ DataTypes[‘’], which is None. With a signle argument: arg -> value, ‘UNKNOWN’-> description. ParameterTypes-> typ_, hex values have integer typ_. Unsuported parameter types will get a NotImplementedError.
- f With two positional arguments: arg1-> value, arg2-> description. ParameterTypes[‘’]-> typ_.
Unsuported parameter types will get a NotImplementedError. With three positional arguments: arg1 casted to DataTypes[arg3]-> value, arg2-> description. arg3-> typ_. Unsuported parameter types will get a NotImplementedError. Incompatible value and typ_ will get a TypeError. Parameters ———-
- __init__(value=None, description='UNKNOWN', typ_='', default=None, valid=None, **kwds)[源代码]
- invoked with no argument results in a parameter of
None value and ‘UNKNOWN’ description ‘’. typ_ DataTypes[‘’], which is None. With a signle argument: arg -> value, ‘UNKNOWN’-> description. ParameterTypes-> typ_, hex values have integer typ_. Unsuported parameter types will get a NotImplementedError.
- f With two positional arguments: arg1-> value, arg2-> description. ParameterTypes[‘’]-> typ_.
Unsuported parameter types will get a NotImplementedError. With three positional arguments: arg1 casted to DataTypes[arg3]-> value, arg2-> description. arg3-> typ_. Unsuported parameter types will get a NotImplementedError. Incompatible value and typ_ will get a TypeError. Parameters ———-
- checked(value)[源代码]
Checks input value against self.type.
If value is none, returns it; else if type is not set, return value after setting type; If value’s type is a subclass of self’s type, return the value; If value’s and self’s types are both subclass of Number, returns value casted in self’s type.
- property default
- setDefault(default)[源代码]
Sets the default of this object.
Default is set directly if type is not set or default is None. If the type of default is not getType(), TypeError is raised.
- setType(typ_)[源代码]
Replaces the current type of this parameter.
Default will be casted if not the same. Unsuported parameter types will get a NotImplementedError.
- setValid(valid)[源代码]
Sets the valid of this object.
If valid is None or empty, set as None, else save in a way so the tuple keys can be serialized with JSON. [[[rangelow, ranehi], state1], [[range2low, r..]..]..]
- setValue(value)[源代码]
Replaces the current value of this parameter.
If value is None set it to None (#TODO: default?) If given/current type is ‘’ and arg value’s type is in DataTypes both value and type are updated to the suitable one in DataTypeNames; or else TypeError is raised. If value type is not a subclass of given/current type, or
Incompatible value and type will get a TypeError.
- split(into=None)[源代码]
split a multiple binary bit-masked parameters according to masks.
into: dictionary mapping bit-masks to the sub-name of the parameter. return: a dictionary mapping name of new parameters to its value.
- string(level=0, alist=False, **kwds)
alist: returns a dictionary string representation of parameter attributes.
- toString(level=0, alist=False, **kwds)[源代码]
alist: returns a dictionary string representation of parameter attributes.
- property valid
- validate(value=Invalid)[源代码]
checks if a match the rule set.
value: will be checked against the ruleset. Default is
self._valid
. returns: (valid value, rule name) for discrete and range rules. {mask: (valid val, rule name, mask_height, mask_width), …} for binary masks rules. (INVALID, ‘Invalid’) if no matching is found. (value, ‘Default’) if rule set is empty.
fdi.dataset.metadataholder module
- class fdi.dataset.metadataholder.MetaDataHolder(meta=None, **kwds)[源代码]
基类:
object
Object holding meta data.
Adds MetaData to the class. with defaults set to self.zInfo[‘metadata’].
- __init__(meta=None, **kwds)[源代码]
Adds MetaData to the class. with defaults set to self.zInfo[‘metadata’].
- getMeta()[源代码]
Returns the current MetaData container of this object. Cannot become a python property because setMeta is in Attributable
fdi.dataset.ndprint module
- fdi.dataset.ndprint.ndprint(data, trans=True, maxElem=9223372036854775807, tablefmt3='plain', **kwds)[源代码]
- makes a formated string of an N-dimensional array for printing.
The fastest changing index is the innerest list. E.g. A 2 by 3 matrix is [[1,2],[3,4],[5,6]] written as:
1 2 3 4 5 6
- But if the matrix is a table, the cells in a column change the fastest,
and the columns are written vertically. So to print a matrix as a table, whose columns are the innerest list, set trans = True (default) then the matrix needs to be printed transposed:
1 3 5 2 4 6
- tablefmt3
control 2d array printing. Default ‘plain’.
fdi.dataset.numericparameter module
- class fdi.dataset.numericparameter.BooleanParameter(value=None, description='UNKNOWN', default=None, valid=None, **kwds)[源代码]
基类:
fdi.dataset.metadata.Parameter
A Parameter that has a boolean as the value.
Set up a parameter whose value is a boolean
- class fdi.dataset.numericparameter.NumericParameter(value=None, description='UNKNOWN', typ_='', default=None, unit=None, valid=None, typecode=None, **kwds)[源代码]
基类:
fdi.dataset.metadata.Parameter
,fdi.dataset.quantifiable.Quantifiable
A Parameter that has a number as the value, a unit, and a typecode.
Set up a parameter whose value is a or a list of numbers.
typ_: type of the parameter value.
fdi.dataset.odict module
- class fdi.dataset.odict.ODict(*args, **kwds)[源代码]
基类:
collections.UserDict
,fdi.dataset.serializable.Serializable
,fdi.dataset.eq.DeepcmpEqual
Ordered dict that is not a subclass of dict and with a better __str__.
- get(name)[源代码]
Raise a
KeyError
to change the default behavior of colections.Mapping to quietly return a None when a key is not found in the dict.
- string(level=0, keyval=None, **kwds)
- level
default=0,
- keyval
default=`None`. If set to a string, ODict class name is not shown and a label of given string is shown with each key as ‘label key:
- ‘.
- tablefmt
=’rst’, tablefmt1=’simple’, tablefmt2=’simple’, matprint=None, trans=True, heavy=True
fdi.dataset.product module
- class fdi.dataset.product.Product(description='UNKNOWN', typ_='Product', level='ALL', creator='UNKNOWN', creationDate=FineTime(Unknown), rootCause='UNKNOWN', version='0.8', FORMATV='1.6.0.10', startDate=FineTime(Unknown), endDate=FineTime(Unknown), instrument='UNKNOWN', modelName='UNKNOWN', mission='_AGS', zInfo=None, **kwds)[源代码]
基类:
fdi.dataset.baseproduct.BaseProduct
Product class schema 1.6 inheriting [‘BaseProduct’].
Automatically generated from fdi/dataset/resources/Product.yml on 2021-12-13 09:38:07.593902.
Description: Project level product
Generally a Product (inheriting BaseProduct) has project-wide attributes and can be extended to define a plethora of specialized products.
Initializes instances with more metadata as attributes, set to default values.
Put description keyword argument here to allow e.g. BaseProduct(“foo”) and description=’foo’
- __init__(description='UNKNOWN', typ_='Product', level='ALL', creator='UNKNOWN', creationDate=FineTime(Unknown), rootCause='UNKNOWN', version='0.8', FORMATV='1.6.0.10', startDate=FineTime(Unknown), endDate=FineTime(Unknown), instrument='UNKNOWN', modelName='UNKNOWN', mission='_AGS', zInfo=None, **kwds)[源代码]
Initializes instances with more metadata as attributes, set to default values.
Put description keyword argument here to allow e.g. BaseProduct(“foo”) and description=’foo’
- property endDate
- property instrument
- property mission
- property modelName
- property startDate
fdi.dataset.quantifiable module
- class fdi.dataset.quantifiable.Quantifiable(unit=None, typecode=None, **kwds)[源代码]
基类:
fdi.dataset.typecoded.Typecoded
A Quantifiable object is a numeric object that has a unit. $ x.unit = ELECTRON_VOLTS $ print x.unit eV [1.60218E-19 J]
Has a unit and a typecode (as in array.array.typecodes).
- __init__(unit=None, typecode=None, **kwds)[源代码]
Has a unit and a typecode (as in array.array.typecodes).
- property unit
fdi.dataset.readonlydict module
- fdi.dataset.readonlydict.ReadOnlyDict
alias of
fdi.dataset.readonlydict.frozendict
- class fdi.dataset.readonlydict.bReadOnlyDict[源代码]
基类:
dict
Dictionary that is read-only and returns all values of dict type to ReadOnlyDict type so they, too, cannot be modified.
Based on https://stackoverflow.com/a/61700719/13472124
- clear(*args, **kwargs)
- pop(*args, **kwargs)
- popitem(*args, **kwargs)
- setdefault(*args, **kwargs)
- update(*args, **kwargs)
- class fdi.dataset.readonlydict.frozendict(*args, **kw)[源代码]
基类:
dict
Extended version of frozent dict.
This extended version of the code above will also “freeze” dictionaries and lists stored as values of the dictionary. If dictionaries/lists are found among the values, they will be handled recursively.
Oren Tirosh, Ero Carrera, Søren Løvborg
https://code.activestate.com/recipes/414283/
- property clear
- property pop
- property popitem
- property setdefault
- property update
fdi.dataset.serializable module
- class fdi.dataset.serializable.Serializable(*args, **kwds)[源代码]
基类:
object
mh: Can be serialized. Has a _STID instance property to show its class information.
- class fdi.dataset.serializable.SerializableEncoder(*, skipkeys=False, ensure_ascii=True, check_circular=True, allow_nan=True, sort_keys=False, indent=None, separators=None, default=None)[源代码]
基类:
json.encoder.JSONEncoder
can encode parameter and product etc such that they can be recovered with deserialize(). Python 3 treats string and unicode as unicode, encoded with utf-8, byte blocks as bytes, encoded with utf-8. Python 2 treats string as str and unicode as unicode, encoded with utf-8, byte blocks as str, encoded with utf-8
Constructor for JSONEncoder, with sensible defaults.
If skipkeys is false, then it is a TypeError to attempt encoding of keys that are not str, int, float or None. If skipkeys is True, such items are simply skipped.
If ensure_ascii is true, the output is guaranteed to be str objects with all incoming non-ASCII characters escaped. If ensure_ascii is false, the output can contain non-ASCII characters.
If check_circular is true, then lists, dicts, and custom encoded objects will be checked for circular references during encoding to prevent an infinite recursion (which would cause an OverflowError). Otherwise, no such check takes place.
If allow_nan is true, then NaN, Infinity, and -Infinity will be encoded as such. This behavior is not JSON specification compliant, but is consistent with most JavaScript based encoders and decoders. Otherwise, it will be a ValueError to encode such floats.
If sort_keys is true, then the output of dictionaries will be sorted by key; this is useful for regression tests to ensure that JSON serializations can be compared on a day-to-day basis.
If indent is a non-negative integer, then JSON array elements and object members will be pretty-printed with that indent level. An indent level of 0 will only insert newlines. None is the most compact representation.
If specified, separators should be an (item_separator, key_separator) tuple. The default is (’, ‘, ‘: ‘) if indent is
None
and (‘,’, ‘: ‘) otherwise. To get the most compact JSON representation, you should specify (‘,’, ‘:’) to eliminate whitespace.If specified, default is a function that gets called for objects that can’t otherwise be serialized. It should return a JSON encodable version of the object or raise a
TypeError
.
- class fdi.dataset.serializable.SerializableEncoderAll(*, skipkeys=False, ensure_ascii=True, check_circular=True, allow_nan=True, sort_keys=False, indent=None, separators=None, default=None)[源代码]
基类:
json.encoder.JSONEncoder
can encode parameter and product etc such that they can be recovered with deserialize(). Python 3 treats string and unicode as unicode, encoded with utf-8, byte blocks as bytes, encoded with utf-8. Python 2 treats string as str and unicode as unicode, encoded with utf-8, byte blocks as str, encoded with utf-8
Constructor for JSONEncoder, with sensible defaults.
If skipkeys is false, then it is a TypeError to attempt encoding of keys that are not str, int, float or None. If skipkeys is True, such items are simply skipped.
If ensure_ascii is true, the output is guaranteed to be str objects with all incoming non-ASCII characters escaped. If ensure_ascii is false, the output can contain non-ASCII characters.
If check_circular is true, then lists, dicts, and custom encoded objects will be checked for circular references during encoding to prevent an infinite recursion (which would cause an OverflowError). Otherwise, no such check takes place.
If allow_nan is true, then NaN, Infinity, and -Infinity will be encoded as such. This behavior is not JSON specification compliant, but is consistent with most JavaScript based encoders and decoders. Otherwise, it will be a ValueError to encode such floats.
If sort_keys is true, then the output of dictionaries will be sorted by key; this is useful for regression tests to ensure that JSON serializations can be compared on a day-to-day basis.
If indent is a non-negative integer, then JSON array elements and object members will be pretty-printed with that indent level. An indent level of 0 will only insert newlines. None is the most compact representation.
If specified, separators should be an (item_separator, key_separator) tuple. The default is (’, ‘, ‘: ‘) if indent is
None
and (‘,’, ‘: ‘) otherwise. To get the most compact JSON representation, you should specify (‘,’, ‘:’) to eliminate whitespace.If specified, default is a function that gets called for objects that can’t otherwise be serialized. It should return a JSON encodable version of the object or raise a
TypeError
.- base = (<class 'str'>, <class 'int'>, <class 'float'>, <class 'bool'>, <class 'NoneType'>)
fdi.dataset.shaped module
fdi.dataset.stringparameter module
- class fdi.dataset.stringparameter.StringParameter(value=None, description='UNKNOWN', default=None, valid=None, typecode='B', **kwds)[源代码]
基类:
fdi.dataset.metadata.Parameter
,fdi.dataset.typecoded.Typecoded
has a unicode string as the value, a typecode for length and char.
- invoked with no argument results in a parameter of
None value and ‘UNKNOWN’ description ‘’. typ_ DataTypes[‘’], which is None. With a signle argument: arg -> value, ‘UNKNOWN’-> description. ParameterTypes-> typ_, hex values have integer typ_. Unsuported parameter types will get a NotImplementedError.
- f With two positional arguments: arg1-> value, arg2-> description. ParameterTypes[‘’]-> typ_.
Unsuported parameter types will get a NotImplementedError. With three positional arguments: arg1 casted to DataTypes[arg3]-> value, arg2-> description. arg3-> typ_. Unsuported parameter types will get a NotImplementedError. Incompatible value and typ_ will get a TypeError. Parameters ———-
fdi.dataset.tabledataset module
- class fdi.dataset.tabledataset.IndexedTableDataset(**kwds)[源代码]
基类:
fdi.dataset.indexed.Indexed
,fdi.dataset.tabledataset.TableDataset
TableDataset with an index table for efficient row look-up.
- vLookUp(key, return_index=True, multiple=False)[源代码]
Similar to Excel VLOOKUP, return all records (rows) that match the key. key: taken as a dictionary key unless
multiple
is True. return_index: if True (default) return index in the array of columns. multiple: if True (default is False) loop through key as a sequence of keys and return a sequece.
- class fdi.dataset.tabledataset.TableDataset(data=None, description=None, typ_=None, version=None, zInfo=None, alwaysMeta=True, **kwds)[源代码]
基类:
fdi.dataset.dataset.CompositeDataset
,fdi.dataset.tabledataset.TableModel
,fdi.dataset.shaped.Shaped
Special dataset that contains a single Array Data object. A TableDataset is a tabular collection of Columns. It is optimized to work on array data.. The column-wise approach is convenient in many cases. For example, one has an event list, and each algorithm is adding a new field to the events (i.e. a new column, for example a quality mask).
Although mechanisms are provided to grow the table row-wise, one should use these with care especially in performance driven environments as this orthogonal approach (adding rows rather than adding columns) is expensive.
General Note:
For reasons of flexibility, memory consumption and performance, this class is not checking whether all columns are of the same length: this is the responsibility of the user/developer. See also the library documentation for more information about this.
Note on column names:
If a column is added without specifying a name, the name ColumnX is created, where X denotes the index of that column. Column name duplicity is not allowed.
Developers:
See “Writing special datasets or products” at the developer’s documentation also.
Please see also this selection example.
- __init__(data=None, description=None, typ_=None, version=None, zInfo=None, alwaysMeta=True, **kwds)[源代码]
- addColumn(name, column, col_des=True)[源代码]
Adds the specified column to this table, and attaches a name to it.
If the name is null, a dummy name “column”+column_count+1 is created, such that it can be accessed by getColumn(str).
If column name exists the corresponding column is substituted.
Parameters: name - column name. column - column to be added. col_des - if True (default) and if column description is ‘UNKNOWN’ or None, set to column name.
- addRow(row, rows=False)[源代码]
Adds the specified map as a new row to this table.
row: mh: row is a dict with names as keys and row data as value. rows: append each element in row if the row data is a list.
- property columnCount
- getColumn(key)[源代码]
Returns the particular column, or a list of columns.
key: ref
getColumnMap()
int/str: return the int-th/named column;
Slice
object, list of columns of sliced column indices;list of integers/strings: return a list of columns corresponding to the given column index/name, or where key is True. Example
a.getColumn(Slice(3,,))
,[2, 4]
,['time', ``energy']
.list of booleans: return a list of columns where key is True. Example
[True, False...]
.
- getColumnMap(key=None)[源代码]
Returns a dict of column-names as the keys and the column(s) as the values.
key: return the following as the value for each key-value pair:
int: name-value where value is the int-th column.
Slice
object, a list of name-columns from slicing the column index. Examplea.getColumn(Slice(3,,))
;Sequence of integers/strings: they are used as the column index/name to select the columns.
Sequence of booleans: columns where the corresponding boolean is True are chosen.
Default is to return all columns.
- getRow(rowIndex)[源代码]
Returns a list containing the objects located in a particular row, or a list of rows.
rowIndex: ref
getRowMap()
int: return the int-th row in a list of elements;
Slice
object, list of integers, list of booleans: return a list of rows each represented by a tuple. Examplea.getRow(Slice(3,,))
,[2,4]
,[True, False...]
.
- getRowMap(rowIndex)[源代码]
Returns a dict of column-names as the keys and the objects located at a particular row(s) as the values.
rowIndex: return the following as the value for each key-value pair:
int: the int-th row’s elements;
Slice
object, a list of rows from slicing the column. Examplea.getRow(Slice(3,,))
;list of integers: they are used as the row index to select the rows.
list of booleans: rows where the corresponding boolean is True are chosen.
- indexOf(key)[源代码]
Returns the index of specified column.
if the key is a Column, it looks for equal references (same column objects), not for equal values. If the key is a string, Returns the index of specified Column name.
- keys()
Returns the column names.
- property list
- removeRow(rowIndex)[源代码]
Removes a row with specified index from this table.
rowIndex: int or a
Slice
object. Examplea.removeRow(Slice(3,,))
. return: removed row data.
- property rowCount
- select(selection)[源代码]
Select a number of rows from this table dataset and return a new TableDataset object containing only the selected rows.
selection: to form a new Tabledataset with ref
getRowMap()
- setColumn(key, value)[源代码]
Replaces a column in this table with specified name to specified column if key is a string and exists, or if the key is an integer in 0 to the number of columns, insert at column-index=key, with the name ‘column’+key, else add a new coolumn.
- setData(data)[源代码]
sets name-column pairs from data.
Valid formd include: {str:Column, …} or [(str, [num, …], str)] or [(str, Column), …] or [[num …], [num …], …]
[{‘name’:str,’column’:Column}] form is deprecated.
Existing data will be discarded except when the provided data is a list of lists, where existing column names and units will remain but data replaced, and extra data items will form new columns named ‘column’+index (index counting from 1) with unit None.
- string(level=0, tablefmt='grid', tablefmt1='simple', tablefmt2='rst', width=0, param_widths=None, matprint=None, trans=True, heavy=True, center=- 1, col_width=8, **kwds)
tablefmt2: format of 2D data, others see MetaData.toString.
- class fdi.dataset.tabledataset.TableModel(**kwds)[源代码]
基类:
object
to interrogate a tabular data model
- getColumnName(columnIndex)[源代码]
Returns the name of the column at columnIndex.
returns a set of columns if key is a slice.
fdi.dataset.tabledataset_datamodel module
fdi.dataset.testproducts module
- class fdi.dataset.testproducts.SP(description='UNKNOWN', typ_='SP', creator='UNKNOWN', version='9', creationDate=FineTime(Unknown), rootCause='UNKNOWN', startDate=FineTime(Unknown), endDate=FineTime(Unknown), instrument='UNKNOWN', modelName='UNKNOWN', mission='_AGS', zInfo=None, **kwds)[源代码]
基类:
fdi.dataset.product.Product
A subclass of Product for tests.
Initializes instances with more metadata as attributes, set to default values.
Put description keyword argument here to allow e.g. BaseProduct(“foo”) and description=’foo’
- class fdi.dataset.testproducts.TP(description='UNKNOWN', typ_='Product', level='ALL', creator='UNKNOWN', creationDate=FineTime(Unknown), rootCause='UNKNOWN', version='0.8', FORMATV='1.6.0.10', startDate=FineTime(Unknown), endDate=FineTime(Unknown), instrument='UNKNOWN', modelName='UNKNOWN', mission='_AGS', zInfo=None, **kwds)[源代码]
基类:
fdi.dataset.product.Product
Initializes instances with more metadata as attributes, set to default values.
Put description keyword argument here to allow e.g. BaseProduct(“foo”) and description=’foo’
- fdi.dataset.testproducts.get_demo_product()[源代码]
|__ meta <MetaData> | |__ description <string> | |__ type <string> | |__ level <string> | |__ creator <string> | |__ creationDate <finetime> | |__ rootCause <string> | |__ version <string> | |__ FORMATV <string> | |__ speed <vector> | __ listeners <ListnerSet> |__ measurements <CompositeDataset> | |__ meta <MetaData> | | __ listeners <ListnerSet> | |__ Time_Energy_Pos <TableDataset> (5, 20) | | |__ meta <MetaData> | | | |__ description <string> | | | |__ shape <tuple> | | | |__ type <string> | | | |__ version <string> | | | |__ FORMATV <string> | | | __ listeners <ListnerSet> | | |__ Time <Column> (20,) | | |__ Energy <Column> (20,) | | |__ Error <Column> (20,) | | |__ y <Column> (20,) | | __ z <Column> (20,) | |__ calibration <ArrayDataset> (11, 11) | __ dset <str> |__ Environment Temperature <ArrayDataset> (7,) |__ Browse <image/png> (5976,) |__ refs <RefContainer> | |__ a_reference <ProductRef> | __ a_different_name <ProductRef> |__ history <History> | |__ PARAM_HISTORY <str> | |__ TASK_HISTORY <str> | __ meta <MetaData> | __ listeners <ListnerSet> __ listeners <ListnerSet>
fdi.dataset.typecoded module
fdi.dataset.typed module
fdi.dataset.unstructureddataset module
- class fdi.dataset.unstructureddataset.UnstrcturedDataset(data=None, description=None, typ_=None, doctype=None, version=None, zInfo=None, alwaysMeta=True, **kwds)[源代码]
基类:
fdi.dataset.dataset.Dataset
,fdi.dataset.copyable.Copyable
Container for data without pre-defined structure or organization..
MetaDataListener must stay to the left of AbstractComposite.
For xmltodict xml_attribs default to
`False`
.- __init__(data=None, description=None, typ_=None, doctype=None, version=None, zInfo=None, alwaysMeta=True, **kwds)[源代码]
- input(data, doctype=None, **kwds)[源代码]
Put data in the dataset.
Depending on doctype: * Default is None for arbitrarily nested Pyton data structure. * Use ‘json’ to have the input string loaded by json.loads, * ‘xml’ by xmltodict.parse.
- jsonPath(expr, val='simple', sep='/', indent=None, *args, **kwds)[源代码]
Make a JSONPath query on the data.
- Expr
JSONPath expression. Ref ‘jsonpath_ng’
- Sep
‘’ or None for keeping jsonpath_ng format (e.g. a.b.[3].d; other string for substituting ‘.’ to the given string, with ‘[’ and ‘]’ removed. Default is ‘/’.
- Val
‘context’ for returning the list of DatumInContext of find; ‘simple’ (default) for list of simple types of values and summarizing list and dict values; other for a list of un-treated `DatumInContext.value`s.
- Indent
for json.dumps.
- 返回
If val is
`context`
, return the list of DatumInContext of jsonpath_ng.ext.parse().find().Else return a list of full_path-value pairs from the output of find().
* If val is
`simple`
, only node values of simple types are kept, list and dict types will show as ‘<list> length’ and ‘<dict> [keys [… [length]]]’, respectively.* If val is
`full`
, the values of returned `list`s are un-treated `DatumInContext.value`s.
fdi.dataset.unstructureddataset_datamodel module
fdi.dataset.vattribute module
fdi.dataset.yaml2python module
- fdi.dataset.yaml2python.dependency_sort(descriptors)[源代码]
sort the descriptors so that everyone’s parents are to his right.
- fdi.dataset.yaml2python.getCls(clp, rerun=True, exclude=None, ignore_error=True, verbose=False)[源代码]
- fdi.dataset.yaml2python.getPython(val, indents, demo, onlyInclude, debug=False)[源代码]
make Model and init__() code strings from given data.
- fdi.dataset.yaml2python.makeinitcode(dt, pval)[源代码]
python instanciation source code.
will be like “default: FineTime1(0)”
- fdi.dataset.yaml2python.params(val, indents, demo, onlyInclude, debug=False)[源代码]
generates python strng for val, a parameter with a set of attribute
val: as in
`name:val`
``` nam:data_type: string description: Description of this dataset default: UNKNOWN valid: ‘’
``` see getPython