Welcome to pdc python api’s documentation!¶
Main APIs¶
Thoughout the documentation, you may see the following types used when an integer must be in a certain range:
type name |
range |
---|---|
int32 |
-2**31 to 2**31-1 |
uint32 |
0 to 2**32-1 |
uint64 |
0 to 2**64-1 |
These bounds are checked at runtime and will result in an OverflowError if they are exceeded.
- class pdc.Type(value)¶
Bases:
Enum
All of the data types that PDC objects support
- INT¶
A 32-bit signed integer. This is the default value for Type aguments
- UINT¶
A 32-bit unsigned integer.
- FLOAT¶
A 32-bit floating point number.
- DOUBLE¶
A 64-bit floating point number.
- INT64¶
A 64-bit signed integer.
- UINT64¶
A 64-bit unsigned integer.
- INT16¶
A 16-bit signed integer.
- INT8¶
An 8-bit signed integer.
- as_numpy_type()¶
Returns the numpy type corresponding to this PDC type
- static from_numpy_type(dtype)¶
Returns the PDC type corresponding to this numpy type
- Parameters:
dtype – The numpy type
- Raises:
ValueError – If the numpy type has no corresponding PDC type
- class pdc.KVTags¶
An object used to manipulate object and container tags. Supports the following operations:
operation
effect
value = tags[x]
get the value of the tag
x
tags[x] = value
set the value of the tag
x
del tags[x]
delete the tag
x
- exception pdc.PDCError¶
A general error type that indicates an error from the underlying c pdc api. Don’t count on getting this error most of the time. Usually, the server/client segfaults instead.
- class pdc.ServerContext¶
A context manager that starts and stops a single PDC server instance on enter and exit. This is intended for testing.
Usage:
with pdc.ServerContext(): pdc.init() ...
- Raises:
FileNotFoundError – if pdc_server.exe is not on PATH
- pdc.init()¶
Initialize PDC. This call will block and eventually terminate the program if the PDC server is not running in the same directory as the program. This must be called before any other PDC method.
Containers¶
- class pdc.Container(self, name: str, lifetime: Lifetime = Lifetime.TRANSIENT)¶
- Containers can contain one or more objects, and objects can be part of one or more containers.Containers can be used to allow objects to persist, and have no other functionality at the moment.Every object must be created belonging to a container.
- Parameters:
name (str) – the name of the container. Container names should be unique between all processes that use PDC.
lifetime (Lifetime) – the container’s lifetime.
- class Lifetime(value)¶
Bases:
Enum
The possible lifetimes of containers.
- Lifetime.PERSISTENT¶
The container and all objects in it will persist across server restarts.
- Lifetime.TRANSIENT¶
The container and all objects in it will be deleted when the server restarts.
- all_local_objects() Iterable[pdc.Object] ¶
Get an iterable through all objects in this container that have been retrieved by the client, and haven’t yet been garbage collected. Deleting an object while this iterator is in use may cause undefined behavior.
- Returns:
An iterable of Objects
- create_object(name: str, properties: Object.Properties)¶
Create a new object, placed in this container. To get an existing object, use Object.get() instead
- Parameters:
name (str) – the name of the object. Object names should be unique between all processes that use PDC, however, passing in a duplicate name will create the object anyway.
properties (ObjectProperties) – An ObjectProperties describing the properties of the object.
- classmethod get(name: str) Container ¶
Get an existing container by name. If you try to get a container that does not exist, the server segfaults and the call blocks indeinitely.
- Parameters:
name (str) – the name of the container.
- Returns:
The container.
- property name: str¶
The name of this container. read-only
- object_from_array(self, name: str, array, type: Optional[Type] = None, prop: Optional['Object.Properties'] = None)¶
Create a new object, placed in this container, from an array-like object. To get an existing object, use Object.get() instead
- Parameters:
name (str) – the name of the object. Object names should be unique between all processes that use PDC, however, passing in a duplicate name will create the object anyway.
array – the array to create the object from. It must be a numpy array, or be convertible to a numpy array via numpy.array().
type – If specified, the resulting object will have this type, if all elements of the array fit in this type.
prop – If specified, the app name, time step, and user id will be copied from this properties object to the resulting object’s properties.
- persist() None ¶
Make this container persistent if it wasn’t already.
- pdc.all_local_containers()
Get an iterable of all containers that have been retrieved by the client, and haven’t yet been garbage collected. Deleting a container after this function is called, then using the iterator may cause undefined behavior.
- Returns:
An iterable of Containers
Objects¶
- class pdc.Object(*args)
A PDC object
- class Properties(self, dims:Union[Tuple[uint32, ...], uint32], type, uint32_t time_step:uint32=0, user_id:Optional[uint32]=None, str app_name:str='')
Contains most of the metadata associated with an object. This is an inner class of Object
- Parameters:
dims (Tuple[uint32, ...] or uint32) – A tuple containing dimensions of the object, or a single integer for 1-D objects. For example, (10, 3) means 10 rows and 3 columns. 1 <= len(dims) <= 2^31-1
type (Type) – the data type.
time_step (uint32) – For applications that involve data along a time axis, this represents the point in time of the data. Defaults to 0.
user_id (uint32) – the id of the user. defaults to os.getuid()
app_name (str) – the name of the application. Defaults to an empty string.
- property app_name: str
The name of the application
- copy() Properties
Copy this properties object
- Returns:
A copy of this object
- property dims: Tuple[int, ...]
The dimensions of this object as a tuple.
- property time_step: uint32
The time step of this object. For applications that involve data along a time axis, this represents the point in time of the data.
- property type: Type
The data type of this object.
- property user_id: uint32
the id of the user.
- class TransferRequest(*args)
Represents a transfer request to either set a region’s data or get it.
- class RequestType(value)
The possible types of transfer request. Can be obtained with
TransferRequest.type
Either GET or SET.
- property done: bool
True if this transfer request is completed
- property result: Optional[ndarray[Any, dtype[ScalarType]]]
If the request is done and the request type is RequestType.GET, this is a numpy array containing the requested data. Otherwise, it is None.
- wait()
Block until the result of the transfer request is available, then return it. If this is a GET request, the return value is the data, otherwise it is None.
- property data: QueryComponent
- An object used to build queries.See queries for more details.
- delete()
Delete this Object. Do not access any methods or properties of this object after calling this.
- property dims: Tuple[uint64, ...]
The dimensions of this object. read-only Equivalent to
obj.get_properties().dims
- classmethod get(name: str) Object
Get an existing object by name
If you have created multiple objects with the same name, this will return the one with time_step=0 If there are multiple objects with the same name and time_step, the tie is broken arbitrarily
- Parameters:
name (str) – the name of the object
- get_data(region: Region = None) TransferRequest
Request a region of data from an object
- Parameters:
region (Region) – the region of data to get. If this is None, defaults to the entire object.
- Returns:
A transfer request representing this request
- Return type:
TransferRequest
- property name: str
The name of this object. read-only
- set_data(self, data, region: 'Region' = None)
Request a region of data from an object to be set to a new value
- Parameters:
region (Region) – the region of data to set. Defaults to the entire object.
data – The data to set the region to. It must be a numpy array or convertible to a numpy array with numpy.array().
- Returns:
A TransferRequest representing this request
- Return type:
TransferRequest
- property type: Type
The type of this object. read-only
Regions¶
- class pdc.region.Region(slice)¶
- A region, which specifies the location of a ‘block’ of data.Regions are created by slicing the
region
object.Examples:all data of an object:region[:]
the first row of a 2 dimensional object, or the first element of a 1 dimensional object:region[0]
the first 3 elements of a 1 dimensional object:region[:3]
the next 3 elements of a 1 dimensional object:region[3:6]
the last 6 columns of the first 2 rows of an object:region[:2, 6:]
The slices used to create a region object may not have step values. (region[1:100:4]
is invalid)- get_absolute(dims)¶
- Return a new absolute region, using the given dimensions to determine ommitted start and stop values.Examples:
region[:].get_absolute((3, 4)) -> region[0:3, 0:4]
region[:2, 6:].get_absolute((10, 10)) -> region[0:2, 6:10]
- is_absolute()¶
- A region is absolute if the region’s slices have both start and stop values.returns True if the region is absoluteExamples:
region[2:3, :9]
is not absoluteregion[2:3, 8]
is absolute
Queries¶
- class pdc.Query(id: uint64, op, left=None, right=None)¶
- class Result(id: pdcid, query: Query)¶
Result of a DataQuery
To get results from this class, index it using objects as keys For example:
query.get_result()[object1]
- property hits¶
Number of hits in the result. If you only want the number of hits, use
query.get_num_hits()
- class pdc.query.QueryComponent(obj: Object)¶
An object used to build a query