150 commits

Author	SHA1	Message	Date
Fabio Manganiello	f45e47363d	Use lazy='joined' instead of lazy='selectin' on Entity.parent. ... That's the best way to ensure that all the columns are fetched eagerly and prevent errors later when trying to access lazily loaded attributes outside of the session/thread.	2023-03-10 12:01:23 +01:00
Fabio Manganiello	8ccf3e804d	Added utility get_entities_engine() function.	2023-03-10 11:49:23 +01:00
Fabio Manganiello	3fcc9957d1	A dirty fix to prevent DetachedInstanceError when accessing the parent entity.	2023-03-10 11:45:44 +01:00
Fabio Manganiello	ceb7a2f098	EntitiesEngine synchronization improvements. ... - Added `wait_start()` method that other threads can use to synchronize with the engine and wait before performing db operations. - Callback logic wrapped in a try/except block to prevent custom integrations with buggy callbacks from crashing the engine.	2023-03-10 00:57:24 +01:00
Fabio Manganiello	73dc2463f1	Added auto_commit=False to entities.save()	2023-03-10 00:40:51 +01:00
Fabio Manganiello	7e92d5f244	Added recursive .copy() method to Entity.	2023-03-09 22:35:31 +01:00
Fabio Manganiello	9f9ee575f1	Added _import_error_ignored_modules. ... ImportErrors on these entity modules will be ignored when dynamically loading them, since they have optional external dependencies and we shouldn't throw an error if we can't import them.	2023-03-09 01:40:35 +01:00
Fabio Manganiello	c4efec6832	Several fixes and improvements on the entities engine. ... - Support for an optional callback on `publish_entities` to get notified when the published object are flushed to the db. - Use `lazy='selectin'` for the entity parent -> children relationship - it is more efficient and it ensures that all the data the application needs is loaded upfront. - `Entity.entity_key` rolled back to `<external_id, plugin>`. The fallback logic on `<id, plugin>` created more problems than those it as supposed to solve. - Added `expire_on_commit=False` to the entities engine session to make sure that we don't get errors on detached/expired instances.	2023-03-09 01:16:04 +01:00
Fabio Manganiello	1781a19a79	s/Entity.to_json/Entity.to_dict/g ... stuff	2023-03-06 23:46:33 +01:00
Fabio Manganiello	b9efa9fa30	entity_key should coalesce (entity.external_id or entity.id)	2023-03-06 16:54:02 +01:00
Fabio Manganiello	ead4513915	Added optional unit column to RawSensor entity.	2023-03-03 01:59:27 +01:00
Fabio Manganiello	2dfb389630	Added remaining bluetooth entity types in _mappers.py.	2023-02-23 21:20:41 +01:00
Fabio Manganiello	a0556d3a42	Added PresenceSensor entities.	2023-02-23 01:42:26 +01:00
Fabio Manganiello	56d693032a	Added DewPointSensor entities.	2023-02-23 01:23:04 +01:00
Fabio Manganiello	d212276247	Added PressureSensor entities.	2023-02-23 01:12:27 +01:00
Fabio Manganiello	d961e2a997	Added TimeDurationSensor entity.	2023-02-23 01:02:13 +01:00
Fabio Manganiello	c3e16f9f9d	Added support for heart rate sensor entities.	2023-02-23 00:55:55 +01:00
Fabio Manganiello	e1b3d52706	Added StepsSensor entity.	2023-02-23 00:45:58 +01:00
Fabio Manganiello	d8c429f4a8	Major improvements on the entities engine. ... - Better logic to recursively link parent/children entities, so partial updates won't get lost. - Removed `EntitiesCache` - it was too much to maintain while keeping consistent with the ORM, and it was a perennial fight against SQLAlchemy's own cache. - Removed `EntityNotifier` - with no need to merge cached entities, the `notify` method has become much simpler and it's simply been merged in the `EntitiesRepository`.	2023-02-22 02:53:45 +01:00
Fabio Manganiello	dc7cbe743d	Refactored/improved RawSensor entity. ... It will now automatically deal with most of the native types and convert them to strings on the db.	2023-02-22 02:19:19 +01:00
Fabio Manganiello	340fd08064	Removed some old type: ignore comments.	2023-02-22 01:29:51 +01:00
Fabio Manganiello	cf219d5a48	Added some more docstrings to entities.	2023-02-22 01:02:26 +01:00
Fabio Manganiello	2fa45fc5a3	Documentation and LINT fixes for sensor entities.	2023-02-21 23:10:05 +01:00
Fabio Manganiello	a6c36fa1c1	Added brand, model and model_id columns to BluetoothDevice.	2023-02-19 23:02:04 +01:00
Fabio Manganiello	72a9a9dfcf	LINT/type fixes	2023-02-19 22:56:45 +01:00
Fabio Manganiello	8aedc3c233	Recursively normalize child entities in EntityManager._normalize_entities	2023-02-18 17:51:57 +01:00
Fabio Manganiello	613e32e7c1	Extended number of supported events and data fields in Bluetooth integration.	2023-02-18 01:15:10 +01:00
Fabio Manganiello	a3aa186ddf	- Added support for scan_pause/scan_resume on bluetooth integration. ... - Added `BluetoothDevice` as its own entity type.	2023-02-13 23:12:25 +01:00
Fabio Manganiello	1d0be5c929	- Simplified prototype for EntityManager.set ... - Added small documentation/annotations notes to the `Plugin` module. - Small LINT fixes	2023-02-11 21:35:00 +01:00
Fabio Manganiello	575635fd6b	Defined set as a base method for all plugins that implement writeable entities	2023-02-11 04:04:21 +01:00
Fabio Manganiello	b0cc80ceb0	Rewriting bluetooth.ble plugin to use bleak instead of gattlib.	2023-02-10 17:40:20 +01:00
Fabio Manganiello	35719b0da9	Let publish_entities return the list of transformed_entities	2023-02-08 02:09:34 +01:00
Fabio Manganiello	a98a5f0980	typo fix	2023-02-08 01:09:25 +01:00
Fabio Manganiello	419a0cec61	More LINTing ... Better prototype for `MultiLevelSwitchEntityManager.set_value`	2023-02-05 23:07:43 +01:00
Fabio Manganiello	06dfd1a152	Added support for more entities in switchbot	2023-02-05 15:34:50 +01:00
Fabio Manganiello	be3b99326f	[WIP] Refactoring @manages annotation into a proper EntityManager hierarchy	2023-02-02 23:21:12 +01:00
Fabio Manganiello	59eb0742a1	s/warnings/logger.debug/ if publish_entities is called with no engine registered	2023-01-29 21:52:12 +01:00
Fabio Manganiello	8aff181956	Merged zwave.mqtt backend into the zwave.mqtt plugin	2023-01-29 02:34:48 +01:00
Fabio Manganiello	afdeb91f66	Implemented remaining supported entities for the smartthings integration	2023-01-26 22:10:02 +01:00
Fabio Manganiello	147f36c86c	All Sensor instances should have is_read_only=True by default	2023-01-22 21:05:14 +01:00
Fabio Manganiello	fd76642082	Added Volume and Muted entities	2023-01-22 21:04:46 +01:00
Fabio Manganiello	dfb13127ee	Added MotionSensor entities	2023-01-21 14:47:18 +01:00
Fabio Manganiello	9a5e2899e8	Support for external_url and image_url on entities	2023-01-15 20:01:47 +01:00
Fabio Manganiello	a58caa17e2	Decreased time of EventQueue (2 -> 1 second) to make entity events more responsive	2023-01-13 23:28:58 +01:00
Fabio Manganiello	38438230d7	The batch of entities currently being processed should have no duplicate keys	2023-01-11 01:22:56 +01:00
Fabio Manganiello	e9c84ff5d4	Support units on dimmer entities	2023-01-09 01:01:05 +01:00
Fabio Manganiello	c32aecece3	Added is_configuration flag to entities	2023-01-03 23:12:27 +01:00
Fabio Manganiello	1c811a490f	Don't serialize children_ids in Entity.to_json() ... Serializing both children_ids and parent_id can result in nasty SQLAlchemy bugs, especially when handling objects that haven't been flushed yet.	2023-01-01 12:47:15 +01:00
Fabio Manganiello	2feaba7bf4	Add children_ids attribute to entities when converted to JSON	2022-12-18 21:03:12 +01:00
Fabio Manganiello	3615a269fe	Use Float instead of Numeric on NumericSensor table. ... The Numeric type may have casting/rounding issues with SQLite.	2022-12-18 15:25:22 +01:00
Fabio Manganiello	5763c5e0ba	Don't use the entities cache when upserting entities. ... This may make things a bit less optimal, but it's probably the only possible solution that preserves my sanity. Managing upserts of cached instances that were previously made transient and expunged from the session is far from easy, and the management of recursive parent/children relationships only add one more layer of complexity (and that management is already complex enough in its current implementation).	2022-12-18 15:13:21 +01:00
Fabio Manganiello	b0464219d3	Large refactor of the entities engine.	2022-12-17 21:41:23 +01:00
Fabio Manganiello	ecba72935f	Check for table metadata existance in Base.metadata instead of having a separate entity registry	2022-12-04 16:28:46 +01:00
Fabio Manganiello	abaeabea22	Implemented recursive merges of parent/child relationships in entities	2022-11-30 00:50:53 +01:00
Fabio Manganiello	0edd73690b	Modelling of parent/children relationships on entity level	2022-11-28 21:36:00 +01:00
Fabio Manganiello	78c59f437a	Added support for illuminance sensor entities	2022-11-27 22:38:58 +01:00
Fabio Manganiello	b9e6614b04	Added support for EnumSensor entities	2022-11-21 00:04:07 +01:00
Fabio Manganiello	69e097707d	Don't lock read session from the main database	2022-11-12 16:10:57 +01:00
Fabio Manganiello	86edd70d93	Fixed session/concurrency management on the main SQLite db ... - The `declarative_base` instance should be shared - Database `session_locks` should be stored at module, not instance level - Better isolation of scoped sessions - Enclapsulated `get_session` method in `UserManager`	2022-11-12 15:36:17 +01:00
Fabio Manganiello	bfeb0a08c4	Encapsulate _get_session in EntityManager	2022-11-12 15:14:10 +01:00
Fabio Manganiello	6b7933cd33	Using a different SQLite database for entities ... This prevents multiprocessing/concurrency issues when modifying the same database file both from the main process and from the web server process	2022-11-12 02:00:55 +01:00
Fabio Manganiello	02a4c9f667	Added is_query_disabled attribute to entities	2022-11-11 20:40:36 +01:00
Fabio Manganiello	00a43dd1f8	Implemented EnumSwitch entity type ... Done for `zigbee.mqtt`, other plugins will follow	2022-11-11 01:46:38 +01:00
Fabio Manganiello	801ed05684	Added support for binary sensors (in zigbee.mqtt for now)	2022-11-05 01:47:50 +01:00
Fabio Manganiello	64513be6b8	Initial implementation of sensor entities. ... Implemented (at least in `zigbee.mqtt`, for now): - `TemperatureSensor` - `HumiditySensor` - `VoltageSensor` - `CurrentSensor` - `EnergySensor` - `PowerSensor` - `NumericSensor` (generic fallback 1) - `RawSensor` (generic fallback 2) - `Sensor` (root class)	2022-11-02 16:38:17 +01:00
Fabio Manganiello	a1cf671334	Added support for link_quality entities to zigbee.mqtt	2022-10-30 11:03:22 +01:00
Fabio Manganiello	691d109fb7	Expunge entities after session commit to ensure that the ORM objects can be reused	2022-10-30 11:00:09 +01:00
Fabio Manganiello	d61b053f72	Support for battery entities	2022-10-29 13:38:42 +02:00
Fabio Manganiello	951950c864	Added dimmer entities	2022-10-23 00:30:32 +02:00
Fabio Manganiello	d7278857e5	Ensure that no records with duplicate key exist within an SQLAlchemy session before flushing	2022-10-23 00:28:42 +02:00
Fabio Manganiello	041f64c80f	Dirty workaround to prevent redefinition of SQLAlchemy ORM model classes	2022-10-10 01:38:15 +02:00
Fabio Manganiello	5f09d449f4	extend_existing=True for entity tables	2022-10-09 23:15:50 +02:00
Fabio Manganiello	6ec8a991df	Fixed tests	2022-10-08 15:18:26 +02:00
Fabio Manganiello	1df71cb54a	Proper support for light entities on smartthings	2022-05-30 09:23:05 +02:00
Fabio Manganiello	30a024befb	Manage hue/sat/bri/ct light ranges on the light entity object itself	2022-04-30 19:38:50 +02:00
Fabio Manganiello	7df67aca82	updated_at should have utcnow() onupdate, not now()	2022-04-30 01:48:55 +02:00
Fabio Manganiello	975d37c562	Added relevant attributes to light entities	2022-04-29 23:29:04 +02:00
Fabio Manganiello	90f067de61	Added reachable flag to device entities	2022-04-29 23:27:35 +02:00
Fabio Manganiello	f45df5d4d3	Explictly cast entity IDs to strings when coalescing entity updates ... Some plugins may represent entity IDs as integers, while the database maps external IDs to strings. This may result in entities being incorrectly mapped during merging. Casting to string prevents these type-related ambiguities.	2022-04-29 23:24:28 +02:00
Fabio Manganiello	47f8520f3b	Added support for description/read_only/write_only on entity level	2022-04-24 22:18:29 +02:00
Fabio Manganiello	ef6b57df31	Added entity info modal and (partial) support for renaming entities	2022-04-23 01:01:14 +02:00
Fabio Manganiello	e6bfa1c50f	Better dynamic entities discovery	2022-04-13 11:25:14 +02:00
Fabio Manganiello	2aa8778078	Do not process EntityUpdateEvents only in case of payload changes ... The UI relies on these events upon refresh to detect if a device is still reacheable. Therefore, we shouldn't mask them if we don't detect any changes with the current entity configuration/state.	2022-04-12 00:41:20 +02:00
Fabio Manganiello	f17245e8c7	Send an EntityUpdateEvent only if an entity has already been persisted ... If an event comes from an entity that hasn't been persisted yet on the internal storage then we wait for the entity record to be committed before firing an event. It's better to wait a couple of seconds for the database to synchronize rather than dealing with entity events with incomplete objects.	2022-04-11 00:38:11 +02:00
Fabio Manganiello	67ff585f6c	Entities engine improvements ... - Added cache support to prevent duplicate EntityUpdateEvents - The cache is smartly pre-populated and kept up-to-date, so it's possible to trigger events as soon as the entities are published by the plugin (not only when the records are flushed to the internal db)	2022-04-11 00:01:21 +02:00
Fabio Manganiello	f52b556219	- icon_class should not be part of the backend model ... - Interaction with entities should occur through the `entities.action` method, not by implementing native methods on each of the model objects	2022-04-08 16:49:47 +02:00
Fabio Manganiello	947b50b937	Added meta as a JSON field on the Entity table ... Metadata attributes can now be defined and overridden on the object itself, as well as on the database. Note that db settings will always take priority in case of value conflicts.	2022-04-07 22:11:31 +02:00
Fabio Manganiello	db7c2095ea	Implemented meta property for entities (for now it only include icon_class)	2022-04-07 18:09:25 +02:00
Fabio Manganiello	d3dc86a5e2	Added documentation for plugin/entity type registry	2022-04-07 01:47:42 +02:00
Fabio Manganiello	28026b0428	Trigger an EntityUpdateEvent when an entity state changes	2022-04-07 01:46:37 +02:00
Fabio Manganiello	44707731a8	Normalize UTC timezone on all the entity timestamps	2022-04-07 01:13:29 +02:00
Fabio Manganiello	948f37afd4	Filter by configured/enabled plugins when returning the entity/plugin registry	2022-04-07 01:04:06 +02:00
Fabio Manganiello	2eeb1d4fea	Entity objects are now JSON-able	2022-04-07 00:21:54 +02:00
Fabio Manganiello	26ffc0b0e1	Use Redis instead of an in-process map to store the entity/plugin registry ... This is particularly useful when we want to access the registry from another process, like the web server or an external script.	2022-04-07 00:18:11 +02:00
Fabio Manganiello	061268cdaf	Support for direct actions on native entities [WIP]	2022-04-05 23:22:54 +02:00
Fabio Manganiello	91ff47167b	Don't terminate the entities engine thread if a batch of entity records fails	2022-04-05 23:04:57 +02:00
Fabio Manganiello	fe0f3202fe	columns should be a property of the Entity object	2022-04-05 23:04:19 +02:00
Fabio Manganiello	8a70f1d38e	Replaced deprecated sqlalchemy.ext.declarative with sqlalchemy.orm	2022-04-05 22:47:44 +02:00
Fabio Manganiello	4b7eeaa4ed	Smarter merging of entities with the same key before they are committed	2022-04-05 21:17:58 +02:00
Fabio Manganiello	4ee7e4db29	Basic support for entities on the local db and implemented support for switch entities on the tplink plugin	2022-04-04 16:50:17 +02:00