exetera.processing package

Submodules

exetera.processing.numpy_buffer module

class exetera.processing.numpy_buffer.ListBuffer(block_pow=8)

Bases: object

append(value)
finalise()
class exetera.processing.numpy_buffer.NumpyBuffer(dtype, block_pow=8)

Bases: object

append(value)
finalise()
class exetera.processing.numpy_buffer.NumpyBuffer2(dtype, block_pow=8)

Bases: object

append(value)
finalise()

exetera.processing.date_time_helpers module

exetera.processing.date_time_helpers.generate_period_offset_map(periods: Sequence[datetime.datetime]) → numpy.ndarray

Given a list of ordered datetimes relating to period boundaries, generate a numpy array of days that map each day to a period.

Example:

[datetime(2020,1,5), datetime(2020,1,12), datatime(2020,1,19), datetime(2020,1,26)]

generates the following output:

[0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2]

In the above example, each period spans a week, and periods cover a total of 3 weeks. As a result, the output is 21 entries long, one for each day covered by the period, and matches each day to the corresponding period.

exetera.processing.date_time_helpers.get_days(date_field: numpy.ndarray, date_filter: Optional[numpy.ndarray] = None, start_date: Optional[numpy.float64] = None, end_date: Optional[numpy.float64] = None) → Tuple[numpy.ndarray, Optional[numpy.ndarray]]

This converts a field of timestamps into a field of relative elapsed days. The precise behaviour depends on the optional parameters but essentially, the lowest valid day is taken as day 0, and all other timestamps are converted to whole numbers of days elapsed since this timestamp:

  • If start_date is set, the start_date is used as the zero-date

  • If start_date is not set:

    • If date_filter is not set, the lowest timestamp is used as the zero-date

    • If date_filter is set, the lowest unfiltered timestamp is used as the zero-date

As well as returning the elapsed days, this method can also return a filter for which elapsed dates are valid. This is determined as follows:

  • If date_filter, start_date and end_date are None, None is returned

  • otherwise:

    • If date_filter is not provided, the filter represents all dates that are out of range with respect to the start_date and end_date parameters

    • If date_filter is provided, the filter is all dates out of range with respect to the start_date and end_date parameters unioned with the date_filter that was passed in

exetera.processing.date_time_helpers.get_period_offsets(periods_by_day: numpy.ndarray, days: numpy.ndarray, in_range: Optional[numpy.ndarray] = None) → numpy.ndarray

Given a periods_by_day, a numpy array of days mapping to periods and days, a numpy array of days to be mapped to periods, perform the mapping to generate a numpy array indicating which period a day is in for each element.

Example:

periods_by_day: [0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2]
days: [3, 18, 4, 7, 10, 0, 0, 2, 19, 20, 16, 17, 19, 4, 5, 9, 8, 15]

generates the following output:

[0, 2, 0, 1, 1, 0, 0, 0, 2, 2, 2, 2, 2, 0, 0, 1, 1, 2]

This function should generally be used in concert with generate_period_offset_map, as follows:

start_date = # a start date
end_date = # an end date
periods = get_periods(start_date, end_date, 'week', 1)

days = get_days(session.get(src['my_table']['my_timestamps']).data[:])
result = get_period_offsets(generate_period_offset_map(periods), days)
exetera.processing.date_time_helpers.get_periods(start_date: datetime.datetime, end_date: datetime.datetime, period: str, delta: int = 1) → Sequence[datetime.datetime]

Generate a set of periods into which timestamped data can be grouped. Delta controls whether the sequence of periods is generated from an start point or an end point. When delta is positive, the sequence is generated forwards in time. When delta is negative, the sequence is generate backwards in time.

Parameters

  • start_date – a datetime.datetime object for the starting period

  • end_date – a datetime.datetime object for tne ending period, exclusive

  • period – a string representing the unit in which the delta is calculated (‘day’, ‘days’, ‘week’, ‘weeks’)

  • delta – an integer representing the delta.

Returns

a list of dates

Module contents