models Package

active_clients Module

Model for cache of Active Client statistics.

class models.active_clients.ActiveClients(*args, **kwds)[source]

Bases: google.appengine.ext.ndb.model.Model

Model that describes information cached about active clients by region.

Notes

Updates to client counts must be written to the datastore. Datastore is considered authoritative for client counts, so updates proceed from the Datastore to Memcache. Memcache is considered authoritative for stats, so updates proceed from Memcache to the Datastore.

geostring[source]
get_stats(client_id, sensor_id, mobile)[source]
last_picked

A Property whose value is any picklable Python object.

mobile_count

A Property whose value is a Python int or long (or bool).

modified

A Property whose value is a datetime object.

Note: Unlike Django, auto_now_add can be overridden by setting the value before writing the entity. And unlike classic db, auto_now does not supply a default value. Also unlike classic db, when the entity is written, the property values are updated to match what was written. Finally, beware that this also updates the value in the in-process cache, and that auto_now_add may interact weirdly with transaction retries (a retry of a property with auto_now_add set will reuse the value that was set on the first try).

resolution

A Property whose value is a Python int or long (or bool).

sensor_stats

A Property whose value is any picklable Python object.

static_count

A Property whose value is a Python int or long (or bool).

total_count

A Property whose value is a Python int or long (or bool).

update(current_time)[source]
update_with_event(client_id, sensor_id, mobile, event_time, event_location, current_time)[source]
class models.active_clients.SensorStats(mobile)[source]

Bases: object

Object that accumulates information about sensors in a region.

Notes

With multiple associators, could store per-sensor-type SensorStats dictionaries so that associators only get the stats for sensors that are relevant to them.

Currently encoding mobility status with stats objects, but could split into multiple dictionaries or encode the keys differently.

events_in_window(start_time, end_time)[source]
events_this_bin(current_time)[source]
to_dict()[source]
update(current_time, modified=False)[source]
update_with_event(current_time, event_time, location)[source]
models.active_clients.get_by_geostring_async(*args, **kwds)[source]

Get an ActiveClients object, or create one, for the specified geostr.

Parameters :

geostr : string

geocell.geostring generated string representing a geographic area.

Returns :

(stats, cached) : (ActiveClients, bool)

Returns the requested ActiveClients object and a boolean indicating whether or not it was cached.

models.active_clients.get_by_location_async(location)[source]

Get an ActiveClients object by an ndb.GeoPt object asynchronously.

models.active_clients.get_geostring(geopt_obj)[source]

Get the geostring for an ndb.GeoPt object.

models.active_clients.get_geostring_by_tuple(location_tuple)[source]

Identical to get_geostring but with a location tuple.

models.active_clients.get_memcache_key(geostr)[source]

Return memcache key name for this geostr.

models.active_clients.get_multi_by_geostring_async(*args, **kwds)[source]

Get multiple ActiveClients objects by their geostrings asynchronously.

Parameters :

geostr_list : list of strings

List of geocell.geostring generated strings to retrieve.

Returns :

active_clients : dict of geostr, ActiveClients

Dictionay mapping requested geostr objects to ActiveClients objects.

models.active_clients.split_stats_key(stats_key)[source]

Return the client id and sensor id components of a stats key.

client Module

Model for client data, including embedded data of all attached sensors.

Includes secret generation and signature validation methods.

class models.client.Client(*args, **kwds)[source]

Bases: google.appengine.ext.ndb.model.Model

Model for client objects.

Notes

Sensors are encoded as ndb.StructuredProperty objects. If the proliferation of sensor data, or large numbers of sensors per client, makes this approach untenable, they could become their own objects with limited changes. For the current use cases, storing them as StructuredProperty objects is more efficient by avoiding unnecessary datastore roundtrips.

Attributes

secret An indexed Property whose value is a text string of limited length.
message_id A Property whose value is a Python int or long (or bool).
created A Property whose value is a datetime object.
last_modified A Property whose value is a datetime object.
name An indexed Property whose value is a text string of limited length.
num_heartbeats A Property whose value is a Python int or long (or bool).
last_ip_address An indexed Property whose value is a text string of limited length.
location A Property whose value is a GeoPt.
geostring An indexed Property whose value is a text string of limited length.
location_source_type A Property whose value is a Python int or long (or bool).
location_description An indexed Property whose value is a text string of limited length.
building An indexed Property whose value is a text string of limited length.
floor An indexed Property whose value is a text string of limited length.
mobility_type A Property whose value is a Python int or long (or bool).
mobile A Property whose value is a Python bool.
software_version An indexed Property whose value is a text string of limited length.
operating_system An indexed Property whose value is a text string of limited length.
uuid An indexed Property whose value is a text string of limited length.
next_sensor_id A Property whose value is a Python int or long (or bool).
sensors A Property whose value is itself an entity.
active A Property whose value is a Python bool.
active_resolution A Property whose value is a Python int or long (or bool).
keyvalue A property whose value is any Json-encodable Python object.
recent_changes A property whose value is any Json-encodable Python object.
data_public A Property whose value is a Python bool.
users An indexed Property whose value is a text string of limited length.
picture A Property whose value is a BlobKey object.
editor_token An indexed Property whose value is a text string of limited length.
editor_expiry A Property whose value is a datetime object.
legacy_identifier An indexed Property whose value is a text string of limited length.
legacy_status A Property whose value is a Python int or long (or bool).
legacy_request_date A Property whose value is a datetime object.
active

A Property whose value is a Python bool.

active_changed

A Property whose value is a datetime object.

Note: Unlike Django, auto_now_add can be overridden by setting the value before writing the entity. And unlike classic db, auto_now does not supply a default value. Also unlike classic db, when the entity is written, the property values are updated to match what was written. Finally, beware that this also updates the value in the in-process cache, and that auto_now_add may interact weirdly with transaction retries (a retry of a property with auto_now_add set will reuse the value that was set on the first try).

active_resolution

A Property whose value is a Python int or long (or bool).

add_sensor(sensor_type)[source]
building

An indexed Property whose value is a text string of limited length.

created

A Property whose value is a datetime object.

Note: Unlike Django, auto_now_add can be overridden by setting the value before writing the entity. And unlike classic db, auto_now does not supply a default value. Also unlike classic db, when the entity is written, the property values are updated to match what was written. Finally, beware that this also updates the value in the in-process cache, and that auto_now_add may interact weirdly with transaction retries (a retry of a property with auto_now_add set will reuse the value that was set on the first try).

data_public

A Property whose value is a Python bool.

editor_expiry

A Property whose value is a datetime object.

Note: Unlike Django, auto_now_add can be overridden by setting the value before writing the entity. And unlike classic db, auto_now does not supply a default value. Also unlike classic db, when the entity is written, the property values are updated to match what was written. Finally, beware that this also updates the value in the in-process cache, and that auto_now_add may interact weirdly with transaction retries (a retry of a property with auto_now_add set will reuse the value that was set on the first try).

editor_token

An indexed Property whose value is a text string of limited length.

floor

An indexed Property whose value is a text string of limited length.

generate_sensor_map()[source]

Create a SensorMap object for this Client.

Returns :

existing_sensors : SensorMap

A dict object mapping Sensor ids to Client.Sensor objects.

geostring

An indexed Property whose value is a text string of limited length.

keyvalue

A property whose value is any Json-encodable Python object.

last_heartbeat

A Property whose value is a datetime object.

Note: Unlike Django, auto_now_add can be overridden by setting the value before writing the entity. And unlike classic db, auto_now does not supply a default value. Also unlike classic db, when the entity is written, the property values are updated to match what was written. Finally, beware that this also updates the value in the in-process cache, and that auto_now_add may interact weirdly with transaction retries (a retry of a property with auto_now_add set will reuse the value that was set on the first try).

last_ip_address

An indexed Property whose value is a text string of limited length.

last_modified

A Property whose value is a datetime object.

Note: Unlike Django, auto_now_add can be overridden by setting the value before writing the entity. And unlike classic db, auto_now does not supply a default value. Also unlike classic db, when the entity is written, the property values are updated to match what was written. Finally, beware that this also updates the value in the in-process cache, and that auto_now_add may interact weirdly with transaction retries (a retry of a property with auto_now_add set will reuse the value that was set on the first try).

legacy_identifier

An indexed Property whose value is a text string of limited length.

legacy_request_date

A Property whose value is a datetime object.

Note: Unlike Django, auto_now_add can be overridden by setting the value before writing the entity. And unlike classic db, auto_now does not supply a default value. Also unlike classic db, when the entity is written, the property values are updated to match what was written. Finally, beware that this also updates the value in the in-process cache, and that auto_now_add may interact weirdly with transaction retries (a retry of a property with auto_now_add set will reuse the value that was set on the first try).

legacy_status

A Property whose value is a Python int or long (or bool).

location

A Property whose value is a GeoPt.

location_description

An indexed Property whose value is a text string of limited length.

location_source_type

A Property whose value is a Python int or long (or bool).

message_id

A Property whose value is a Python int or long (or bool).

metadata_changed

A Property whose value is a Python bool.

mobile

A Property whose value is a Python bool.

mobility_type

A Property whose value is a Python int or long (or bool).

name

An indexed Property whose value is a text string of limited length.

next_sensor_id

A Property whose value is a Python int or long (or bool).

num_heartbeats

A Property whose value is a Python int or long (or bool).

operating_system

An indexed Property whose value is a text string of limited length.

picture

A Property whose value is a BlobKey object.

recent_changes

A property whose value is any Json-encodable Python object.

secret

An indexed Property whose value is a text string of limited length.

sensors

A Property whose value is itself an entity.

The values of the sub-entity are indexed and can be queried.

See the module docstring for details.

software_version

An indexed Property whose value is a text string of limited length.

users

An indexed Property whose value is a text string of limited length.

uuid

An indexed Property whose value is a text string of limited length.

class models.client.EventRate(*args, **kwds)[source]

Bases: google.appengine.ext.ndb.model.Model

Model that describes the noisiness of the sensor.

Attributes

last_updated A Property whose value is a datetime object.
time_window A Property whose value is a Python int or long (or bool).
num_events ndb.IntegerProperty Number of events in time_window seconds.
event_count

A Property whose value is a Python int or long (or bool).

last_updated

A Property whose value is a datetime object.

Note: Unlike Django, auto_now_add can be overridden by setting the value before writing the entity. And unlike classic db, auto_now does not supply a default value. Also unlike classic db, when the entity is written, the property values are updated to match what was written. Finally, beware that this also updates the value in the in-process cache, and that auto_now_add may interact weirdly with transaction retries (a retry of a property with auto_now_add set will reuse the value that was set on the first try).

time_window

A Property whose value is a Python int or long (or bool).

class models.client.Sensor(*args, **kwds)[source]

Bases: google.appengine.ext.ndb.model.Model

Model for sensor specific data that should be indexed.

Notes

Some of the Sensor data does not strictly need indexed, but changes slowly enought that it is not concerning that it is indexed. The indexed parameter cannot be specified for StructuredProperty objects, so moving attributes that don’t need indexed to a LocalStructuredProperty, like the SensorStatus object, is one way to have unindexed properties in a StructuredProperty object.

Attributes

sensor_id A Property whose value is a Python int or long (or bool).
sensor_type A Property whose value is a Python int or long (or bool).
created A Property whose value is a datetime object.
calibrated A Property whose value is a Python bool.
units An indexed Property whose value is a text string of limited length.
num_samples A Property whose value is a Python int or long (or bool).
sample_window_size A Property whose value is a Python int or long (or bool).
model An indexed Property whose value is a text string of limited length.
serial An indexed Property whose value is a text string of limited length.
connected A Property whose value is a Python bool.
removed A Property whose value is a Python bool.
request_all_sensor_data A Property whose value is a Python bool.
event_rate Substructure that is serialized to an opaque blob.
picture A Property whose value is a BlobKey object.
calibrated

A Property whose value is a Python bool.

connected

A Property whose value is a Python bool.

created

A Property whose value is a datetime object.

Note: Unlike Django, auto_now_add can be overridden by setting the value before writing the entity. And unlike classic db, auto_now does not supply a default value. Also unlike classic db, when the entity is written, the property values are updated to match what was written. Finally, beware that this also updates the value in the in-process cache, and that auto_now_add may interact weirdly with transaction retries (a retry of a property with auto_now_add set will reuse the value that was set on the first try).

event_rate

Substructure that is serialized to an opaque blob.

This looks like StructuredProperty on the Python side, but is written like a BlobProperty in the datastore. It is not indexed and you cannot query for subproperties. On the other hand, the on-disk representation is more efficient and can be made even more efficient by passing compressed=True, which compresses the blob data using gzip.

model

An indexed Property whose value is a text string of limited length.

num_samples

A Property whose value is a Python int or long (or bool).

picture

A Property whose value is a BlobKey object.

removed

A Property whose value is a Python bool.

request_all_sensor_data

A Property whose value is a Python bool.

sample_window_size

A Property whose value is a Python int or long (or bool).

sensor_id

A Property whose value is a Python int or long (or bool).

sensor_type

A Property whose value is a Python int or long (or bool).

serial

An indexed Property whose value is a text string of limited length.

units

An indexed Property whose value is a text string of limited length.

class models.client.SensorMap(*args)[source]

Bases: dict

Dictionary object mapping Sensor ids to Client.Sensor objects.

Raises :

UnknownSensor :

Raised when an invalid sensor id is requested from the dictionary.

exception models.client.UnknownSensor[source]

Bases: exceptions.Exception

models.client.generate_secret(environment, current_time)[source]

Create a random value using the system environment and time as a seed.

Parameters :

environment : webapp2.environment

Used to provide additional entropy when hashing with the system time.

Notes

This generator should be sufficiently random for our limited purposes, but PyCrypto is available as a Python 2.7 library on App Engine if additional security is desired.

models.client.signature_valid(client, signature_pb)[source]

Verifies signature from client matches provided details and stored secret.

Also guarantees that message_id moves, over time.

Parameters :

client : Client

Client object to retrieve secret and message_id from.

signature_pb : messages.common_pb2.ClientSignature

Protocol buffer object to validate.

Returns :

valid : bool

True if provided signature appears to have been provided by the identified Client.

Notes

Automatically sets the latest message_id of the Client to the provided message_id in the protocol buffer. It is fine if this temporarily moves the latest message_id backward: it prevents the repeated replay of the current message_id.

Always except messages with more a more recent message_id, but limit how far back in time / out of order a message can be before it is rejected. The limit is controlled by MAX_MESSAGE_ID_DIFF.

event Module

Module to manage event objects added to and retrieved from memcache.

class models.event.CachedEvent

Bases: tuple

client_id

Alias for field number 2

date_str

Alias for field number 0

event_count

Alias for field number 7

location

Alias for field number 1

mobile

Alias for field number 3

readings

Alias for field number 6

sensor_id

Alias for field number 4

sensor_type

Alias for field number 5

time_window

Alias for field number 8

models.event.add_sensor_event_to_cache_async(*args, **kwds)[source]

Convert event details to a tuple to store in memcache.

Notes

Use of an internal class more than doubles the serialized size for little benefit. Only this method and the EventListHandler should ever touch the serialized form, so the efficiency gains for the most numerous object seem worth the limited inconvenience. Passing the serialized tuple to CachedEvent buys named fields and checks the length of the tuple, but little else.

models.event.get_event_index()[source]
models.event.get_events_from_cache(last_index)[source]

global_state Module

sensor_reading Module

Contains SensorReading class and related methods.

class models.sensor_reading.SensorReading(*args, **kwds)[source]

Bases: google.appengine.ext.ndb.model.Model

Model for sensor reading metadata. Links to blobstore content.

Attributes

created A Property whose value is a datetime object.
ongoing_data A Property whose value is a Python bool.
request_id A Property whose value is a Python int or long (or bool).
client_id A Property whose value is a Python int or long (or bool).
sensor_id A Property whose value is a Python int or long (or bool).
start_date A Property whose value is a datetime object.
end_date A Property whose value is a datetime object.
location A Property whose value is a GeoPt.
data_format A Property whose value is a Python int or long (or bool).
data_key A Property whose value is a BlobKey object.
client_id

A Property whose value is a Python int or long (or bool).

created

A Property whose value is a datetime object.

Note: Unlike Django, auto_now_add can be overridden by setting the value before writing the entity. And unlike classic db, auto_now does not supply a default value. Also unlike classic db, when the entity is written, the property values are updated to match what was written. Finally, beware that this also updates the value in the in-process cache, and that auto_now_add may interact weirdly with transaction retries (a retry of a property with auto_now_add set will reuse the value that was set on the first try).

data_format

A Property whose value is a Python int or long (or bool).

data_key

A Property whose value is a BlobKey object.

end_date

A Property whose value is a datetime object.

Note: Unlike Django, auto_now_add can be overridden by setting the value before writing the entity. And unlike classic db, auto_now does not supply a default value. Also unlike classic db, when the entity is written, the property values are updated to match what was written. Finally, beware that this also updates the value in the in-process cache, and that auto_now_add may interact weirdly with transaction retries (a retry of a property with auto_now_add set will reuse the value that was set on the first try).

expires

A Property whose value is a Python bool.

generate_filename()[source]
location

A Property whose value is a GeoPt.

ongoing_data

A Property whose value is a Python bool.

public

A Property whose value is a Python bool.

request_id

A Property whose value is a Python int or long (or bool).

sensor_id

A Property whose value is a Python int or long (or bool).

set_location(latitude, longitude)[source]

Avoids requiring ndb import in other modules.

start_date

A Property whose value is a datetime object.

Note: Unlike Django, auto_now_add can be overridden by setting the value before writing the entity. And unlike classic db, auto_now does not supply a default value. Also unlike classic db, when the entity is written, the property values are updated to match what was written. Finally, beware that this also updates the value in the in-process cache, and that auto_now_add may interact weirdly with transaction retries (a retry of a property with auto_now_add set will reuse the value that was set on the first try).

user Module

Model for user data.

class models.user.User(*args, **kwds)[source]

Bases: google.appengine.ext.ndb.model.Model

Model for user specific information.

admin

A Property whose value is a Python bool.

avatar_url

An indexed Property whose value is a text string of limited length.

clients

A Property whose value is a Python int or long (or bool).

created

A Property whose value is a datetime object.

Note: Unlike Django, auto_now_add can be overridden by setting the value before writing the entity. And unlike classic db, auto_now does not supply a default value. Also unlike classic db, when the entity is written, the property values are updated to match what was written. Finally, beware that this also updates the value in the in-process cache, and that auto_now_add may interact weirdly with transaction retries (a retry of a property with auto_now_add set will reuse the value that was set on the first try).

email

An indexed Property whose value is a text string of limited length.

email_token

An indexed Property whose value is a text string of limited length.

email_verified

A Property whose value is a Python bool.

last_ip_address

An indexed Property whose value is a text string of limited length.

An indexed Property whose value is a text string of limited length.

modified

A Property whose value is a datetime object.

Note: Unlike Django, auto_now_add can be overridden by setting the value before writing the entity. And unlike classic db, auto_now does not supply a default value. Also unlike classic db, when the entity is written, the property values are updated to match what was written. Finally, beware that this also updates the value in the in-process cache, and that auto_now_add may interact weirdly with transaction retries (a retry of a property with auto_now_add set will reuse the value that was set on the first try).

name

An indexed Property whose value is a text string of limited length.

class models.user.UserToken(*args, **kwds)[source]

Bases: google.appengine.ext.ndb.model.Model

Model for User tokens.

daemon

A Property whose value is a Python bool.

expires

A Property whose value is a datetime object.

Note: Unlike Django, auto_now_add can be overridden by setting the value before writing the entity. And unlike classic db, auto_now does not supply a default value. Also unlike classic db, when the entity is written, the property values are updated to match what was written. Finally, beware that this also updates the value in the in-process cache, and that auto_now_add may interact weirdly with transaction retries (a retry of a property with auto_now_add set will reuse the value that was set on the first try).

ip_address

An indexed Property whose value is a text string of limited length.

login_time

A Property whose value is a datetime object.

Note: Unlike Django, auto_now_add can be overridden by setting the value before writing the entity. And unlike classic db, auto_now does not supply a default value. Also unlike classic db, when the entity is written, the property values are updated to match what was written. Finally, beware that this also updates the value in the in-process cache, and that auto_now_add may interact weirdly with transaction retries (a retry of a property with auto_now_add set will reuse the value that was set on the first try).

user

A Property whose value is a Key object.

Optional keyword argument: kind=<kind>, to require that keys assigned to this property always have the indicated kind. May be a string or a Model subclass.

user_agent

An indexed Property whose value is a text string of limited length.

validated

A Property whose value is a datetime object.

Note: Unlike Django, auto_now_add can be overridden by setting the value before writing the entity. And unlike classic db, auto_now does not supply a default value. Also unlike classic db, when the entity is written, the property values are updated to match what was written. Finally, beware that this also updates the value in the in-process cache, and that auto_now_add may interact weirdly with transaction retries (a retry of a property with auto_now_add set will reuse the value that was set on the first try).

models.user.create_user(*args, **kwds)[source]
models.user.generate_token(*args, **kwds)[source]
models.user.get_user(auth_id)[source]
models.user.mark_token_as_daemon(*args, **kwds)[source]
models.user.save_user_changes(user, user_dict)[source]
models.user.validate_token(user_id, token_str, current_time, ip_address, user_agent, no_renew=False)[source]

Validate provided token and return user and a valid token.

Regenerates a new token after TOKEN_RENEW_AGE has elapsed.

Table Of Contents

Previous topic

messages Package

This Page