Welcome to pdc python api’s documentation!

Main APIs

Thoughout the documentation, you may see the following types used when an integer must be in a certain range:

type name

range

int32

-2**31 to 2**31-1

uint32

0 to 2**32-1

uint64

0 to 2**64-1

These bounds are checked at runtime and will result in an OverflowError if they are exceeded.

class pdc.Type(value)

Bases: Enum

All of the data types that PDC objects support

INT

A 32-bit signed integer. This is the default value for Type aguments

UINT

A 32-bit unsigned integer.

FLOAT

A 32-bit floating point number.

DOUBLE

A 64-bit floating point number.

INT64

A 64-bit signed integer.

UINT64

A 64-bit unsigned integer.

INT16

A 16-bit signed integer.

INT8

An 8-bit signed integer.

as_numpy_type()

Returns the numpy type corresponding to this PDC type

static from_numpy_type(dtype)

Returns the PDC type corresponding to this numpy type

Parameters:

dtype – The numpy type

Raises:

ValueError – If the numpy type has no corresponding PDC type

class pdc.KVTags

An object used to manipulate object and container tags. Supports the following operations:

operation

effect

value = tags[x]

get the value of the tag x

tags[x] = value

set the value of the tag x

del tags[x]

delete the tag x
exception pdc.PDCError

A general error type that indicates an error from the underlying c pdc api. Don’t count on getting this error most of the time. Usually, the server/client segfaults instead.

class pdc.ServerContext

A context manager that starts and stops a single PDC server instance on enter and exit. This is intended for testing.

Usage:

with pdc.ServerContext():
    pdc.init()
    ...
Raises:

FileNotFoundError – if pdc_server.exe is not on PATH

pdc.init()

Initialize PDC. This call will block and eventually terminate the program if the PDC server is not running in the same directory as the program. This must be called before any other PDC method.

pdc.ready()

True if init() has been called, False otherwise

Containers

class pdc.Container(self, name: str, lifetime: Lifetime = Lifetime.TRANSIENT)
Containers can contain one or more objects, and objects can be part of one or more containers.
Containers can be used to allow objects to persist, and have no other functionality at the moment.
Every object must be created belonging to a container.
Parameters:

  • name (str) – the name of the container. Container names should be unique between all processes that use PDC.

  • lifetime (Lifetime) – the container’s lifetime.

class Lifetime(value)

Bases: Enum

The possible lifetimes of containers.

Lifetime.PERSISTENT

The container and all objects in it will persist across server restarts.

Lifetime.TRANSIENT

The container and all objects in it will be deleted when the server restarts.

all_local_objects() → Iterable[pdc.Object]

Get an iterable through all objects in this container that have been retrieved by the client, and haven’t yet been garbage collected. Deleting an object while this iterator is in use may cause undefined behavior.

Returns:

An iterable of Objects

create_object(name: str, properties: Object.Properties)

Create a new object, placed in this container. To get an existing object, use Object.get() instead

Parameters:

  • name (str) – the name of the object. Object names should be unique between all processes that use PDC, however, passing in a duplicate name will create the object anyway.

  • properties (ObjectProperties) – An ObjectProperties describing the properties of the object.

classmethod get(name: str) → Container

Get an existing container by name. If you try to get a container that does not exist, the server segfaults and the call blocks indeinitely.

Parameters:

name (str) – the name of the container.

Returns:

The container.

property lifetime: Lifetime

The lifetime of this container. read-only.

property name: str

The name of this container. read-only

object_from_array(self, name: str, array, type: Optional[Type] = None, prop: Optional['Object.Properties'] = None)

Create a new object, placed in this container, from an array-like object. To get an existing object, use Object.get() instead

Parameters:

  • name (str) – the name of the object. Object names should be unique between all processes that use PDC, however, passing in a duplicate name will create the object anyway.

  • array – the array to create the object from. It must be a numpy array, or be convertible to a numpy array via numpy.array().

  • type – If specified, the resulting object will have this type, if all elements of the array fit in this type.

  • prop – If specified, the app name, time step, and user id will be copied from this properties object to the resulting object’s properties.

persist() → None

Make this container persistent if it wasn’t already.

property tags: KVTags

The tags of this container. See KVTags for more information.

NOTE: currently, deleting tags from containers is not supported.

pdc.all_local_containers()

Get an iterable of all containers that have been retrieved by the client, and haven’t yet been garbage collected. Deleting a container after this function is called, then using the iterator may cause undefined behavior.

Returns:

An iterable of Containers

Objects

class pdc.Object(*args)

A PDC object

class Properties(self, dims:Union[Tuple[uint32, ...], uint32], type, uint32_t time_step:uint32=0, user_id:Optional[uint32]=None, str app_name:str='')

Contains most of the metadata associated with an object. This is an inner class of Object

Parameters:

  • dims (Tuple[uint32, ...] or uint32) – A tuple containing dimensions of the object, or a single integer for 1-D objects. For example, (10, 3) means 10 rows and 3 columns. 1 <= len(dims) <= 2^31-1

  • type (Type) – the data type.

  • time_step (uint32) – For applications that involve data along a time axis, this represents the point in time of the data. Defaults to 0.

  • user_id (uint32) – the id of the user. defaults to os.getuid()

  • app_name (str) – the name of the application. Defaults to an empty string.

property app_name: str

The name of the application

copy() → Properties

Copy this properties object

Returns:

A copy of this object

property dims: Tuple[int, ...]

The dimensions of this object as a tuple.

property time_step: uint32

The time step of this object. For applications that involve data along a time axis, this represents the point in time of the data.

property type: Type

The data type of this object.

property user_id: uint32

the id of the user.

class TransferRequest(*args)

Represents a transfer request to either set a region’s data or get it.

class RequestType(value)

The possible types of transfer request. Can be obtained with TransferRequest.type Either GET or SET.

property done: bool

True if this transfer request is completed

property result: Optional[ndarray[Any, dtype[ScalarType]]]

If the request is done and the request type is RequestType.GET, this is a numpy array containing the requested data. Otherwise, it is None.

wait()

Block until the result of the transfer request is available, then return it. If this is a GET request, the return value is the data, otherwise it is None.

property data: QueryComponent
An object used to build queries.
See queries for more details.
delete()

Delete this Object. Do not access any methods or properties of this object after calling this.

property dims: Tuple[uint64, ...]

The dimensions of this object. read-only Equivalent to obj.get_properties().dims

classmethod get(name: str) → Object

Get an existing object by name

If you have created multiple objects with the same name, this will return the one with time_step=0 If there are multiple objects with the same name and time_step, the tie is broken arbitrarily

Parameters:

name (str) – the name of the object

get_data(region: Region = None) → TransferRequest

Request a region of data from an object

Parameters:

region (Region) – the region of data to get. If this is None, defaults to the entire object.

Returns:

A transfer request representing this request

Return type:

TransferRequest

property name: str

The name of this object. read-only

set_data(self, data, region: 'Region' = None)

Request a region of data from an object to be set to a new value

Parameters:

  • region (Region) – the region of data to set. Defaults to the entire object.

  • data – The data to set the region to. It must be a numpy array or convertible to a numpy array with numpy.array().

Returns:

A TransferRequest representing this request

Return type:

TransferRequest

property type: Type

The type of this object. read-only

Regions

property pdc.region.region

An object used to create regions. See Region for details.

class pdc.region.Region(slice)
A region, which specifies the location of a ‘block’ of data.
Regions are created by slicing the region object.
Examples:
all data of an object: region[:]
the first row of a 2 dimensional object, or the first element of a 1 dimensional object: region[0]
the first 3 elements of a 1 dimensional object: region[:3]
the next 3 elements of a 1 dimensional object: region[3:6]
the last 6 columns of the first 2 rows of an object: region[:2, 6:]

The slices used to create a region object may not have step values. (region[1:100:4] is invalid)
get_absolute(dims)
Return a new absolute region, using the given dimensions to determine ommitted start and stop values.
Examples:
region[:].get_absolute((3, 4)) -> region[0:3, 0:4]
region[:2, 6:].get_absolute((10, 10)) -> region[0:2, 6:10]
is_absolute()
A region is absolute if the region’s slices have both start and stop values.
returns True if the region is absolute
Examples:
region[2:3, :9] is not absolute
region[2:3, 8] is absolute

Queries

class pdc.Query(id: uint64, op, left=None, right=None)
class Result(id: pdcid, query: Query)

Result of a DataQuery

To get results from this class, index it using objects as keys For example: query.get_result()[object1]

property hits

Number of hits in the result. If you only want the number of hits, use query.get_num_hits()

get_num_hits(region: Region = None) → int

Get number of hits for this query (i.e. number of elements that match this query)

Parameters:

region (Region) – The region to query over. If this is None, the entire object is queried.

Returns:

number of hits

get_result(region: Region = None) → Result

Get the result of this query

Parameters:

region (Region) – The region to query over. If this is None, the entire object is queried.

Returns:

the Result object

class pdc.query.QueryComponent(obj: Object)

An object used to build a query

Indices and tables