Add stand-alone 'energenie' command to switch devices on & off.

Small fixes for Python 3

Tested MIHO014 relay on Raspberry pi - works fine

Updated README with better links to places to get the code from

Merge pull request #74 from whaleygeek/ook_radio_config ...

Ook radio config

Branch ready for merging back to master

Tested with new devices on Raspberry Pi. All work OK

Added in support for other devices like control and monitor only

Added support for MIHO014 in line relay. ...

Tested in simulation

inner_times now correctly set on both interfaces. ...

Tested in simulation

Now passes and uses inner_times override to radio driver ...

MIHO008 white_light working in simulation
and the ENER002 still uses the older inner_times=8

radio_config now making it through to OnAir.TwoBitInterface.send()

Device.config is now Device.radio_config ...

This is to prevent confusion with get_config and get_radio_config

Clearly documented plan of steps to implement the change

Documented work required to get the MiHome Light integrated

Merge pull request #73 from whaleygeek/new_devices_june16 ...

New devices june16

Reviewed TODO items for new device support

Tested MIHO032 PIR sensor, works fine

Added support for MIHO032 MotionSensor

Support for MIHO003 OpenSensor tested and working

Added MIHO033 OpenSensor to Devices.py

Documented messages from 3 new devices

Spec'ed out work required to add the new devices

Now runs on Raspberry Pi 3 also

working notes file device_classes_branch.txt moved over to an issue

Reviewed TODO list from the device_classes branch ...

The remaining items will be moved to an issue for later mop up

Removed sample programs and moved to issues in the issue log ...

These are not ready to be released yet and are at early ideas stage.

Updated README.md to bring in line with latest release

Merge pull request #61 from whaleygeek/device_classes ...

Device classes

New tests documented. ...

All tests done and pass

Tested and tweaked for Python 3 support

Tested demos on real hardware. Works fine. ...

Observation: handling of incoming messages does not check registry,
it only checks the router, so if you don't have a device object
bound to it, it will be seen as unknown. Needs fixing, but not a high
priority.

Tested setup_tool.py on real hardware - works OK ...

Some tweaks required but now works fine

All test cases now run fine. ...

There is no capture/compare at the moment, so verification is manual

All testers except Registry_test now running all tests

Rationalised all comment styles. ...

The double hash (##) is reserved for commented out lines of code.
The single hash (#) is reserved for normal comments.

Moved setup code from global scope of Registry to energenie.init() ...

This makes it possible to use the registry in test cases

auto variable create example added. ...

Tested in simulation, seems to work, but need to re-test on hardware

Energy monitor example written. ...

Not tested yet, as need to test with real devices

Standalone discovery example written. ...

Not able to test until real devices added

Improved registry listing by using iterator

Iterator added to registry

Clarified control_any_noreg.py by adding a MIHO005 example

Renamed a lot of the control_ demos, due to device agnostic behaviour

Removed control_both.py, as control_any.py will do any sockets now

control_mihome.py recoded to use remembered registry

control_legacy.py simplified, assuming learning done by setup_tool.py

setup_tool tested in simulation and works OK

setup_tool finished, ready for testing

setup_tool now written, ready to start testing

About half of the setup tool is written and roughly working

Design for all setup_tool actions done

setup_tool main menu written and working

Defined requirements for demo apps including setup tool menu

Specified the demos that we want to write as part of the device classes testing

Receive counter implemented and tested

Devices_test is now a unittest

All registry tests now pass. ...

Requirements list reviewed.
TODO list updated for next steps

Just the route creator to test now

Most of Registry_test completed. ...

Just need to write code that passes the 'rx routes created' test

KVS all test cases pass

KVS.DEL() passing tests

Factory creation in ADD is now working

kvs remove functionality working

KVS.remove() test now passes

KVS tested in non persisted mode and works. ...

KVS mostly tested in persisted mode, but more code to write and test

KVS tests for non persistent KVS all pass

Test harness and functionality for KVS defined

Moved KVS to separate module with separate test harness. ...

Will test this in isolation first before integrating with Registry

Managed to persist device configuration to registry.kvs

Beginnings of a persistent version of a class is appearing

Test harness for new Registry and KVS in place

Design of persistent registry store improved after a re-think

Pseudo code for KVS written, ready to be coded up

Design for Registry data store (KVS) done

Thought through next set of demos

monitor_mihome now works with auto join and switch toggling

Factory method added to MiHomeDevice to generate a join_req for testing

Registry_test all test cases now pass for discovery

Set up test case for the final discovery test. ...

Test not run yet, and code is missing, marked with ####HERE####

Join_ack working on discovery mode autojoin

Most test cases for discovery modes implemented and passing

Auto discovery tested and working with accept

Initial outer testing of 4 discovery modes done ok

Framework of discovery methods in place, ready to TDD

Reviewed discovery user requirements, updated todo list

Design for discovery agents done

Updated plan for next steps

Can now construct a device class instance from its product_id

Default incoming message callback added to router, for logging/debug

Router unknown message handling callback registration is cleaner

Neater callback mechanism for device class instances to use. ...

This new architecture ensures default message processing always happens

Fixed bug where sensorid was always zero in csv log

Synthetic receive working in monitor_mihome.py

Updated TODO list with next steps for receive pipeline

energenie.loop() written with FSK receiver connected to fsk_router

Router ready to be tied into a loop()

control_both.py working in simulation

switch.py renamed as control_mihome.py, rewritten for new OO interface

legacy.py rewritten to use new device class interface

Tested on Raspberry Pi, all 4 testers still work with new Message()

Updated TODO list for the next phase of work on device_classes branch ...

Specifically re-testing samples, then pushing OO classes up into app.

Improved debug output for testing

Ran all test cases again (no hardware), fixed a few issues. ...

Need to still run this up on the hardware to test properly

Code now ready for regression testing with new Message() wrappers added

Refactored usages of raw message to use Message() ...

Removed all usages of showMessage, alterMessage

Marked with ##HERE markers the refactoring required for new Message() ...

Disabled alterMessage showMessage and createMessage to force this issue

OpenThings.Message tests now completed

PARAM_NAME key syntax for OpenThings.Message [set] tested ok

OpenThings.Message now tests attribute set and get. ...

Added decorators for @test_0 and @test_1 to turn tests on and off selectively

setattr tests done on OpenThings.Message

Tested all attribute read services in OpenThings.Message

OpenThings.Message test harness keyed paths tested and working

Characterised usage modes of OpenThings.Message better

Refinements to different parameter formats

Clarified test cases for OpenThings.Message()

Started writing unittest harness for new OpenThings.Message abstraction

Message.get() implemented and tested with paramid keying

Started to think through a better message abstraction

Started to think through a better alterMessage ...

characters that can be used in an identifier need checking

Improved separation of test harness messages

Major tidy up of test harness and lifecycle methods

Moved test harnesses into separate _test.py modules

More representative synthetic data added to FSK receive test

Poked a synthetic switch on message down the MIHO005 receive pipeline

Started to flesh out the MiHome receive pipeline

Spec'ed interface for OOK decoder

Deprecated Messages due to import race with Devices. ...

Template messages (which are cloned with create_message) now in Devices.

Commissioned a tx path for MiHome adaptor Plus. ...

This is a temporary path, but it generates the correct message

LegacyDevice tx path commissioned end to end

Most of the OnAir interface coded or spec'ed now

Implementation checklist added for OnAir

Method signatures defined for OnAir adaptors

Spec'ed out responsibilities of OnAir interfaces

Placeholder added to architecture for radio interface adaptors

More design on the tx/rx and air_interface pipeline

Renamed encoder as TwoBit, as it encodes two bits per byte. ...

This is the encoding protocol used with legacy OOK devices

Started to plan out the tx/rx pipeline for payloads and message buffers

Initial design of a message Router

Capabilities of devices turned into flags, with query method in base class

Mock registry store created, auto_create is working

Documented requirement for a simple persistent data store for the Registry

Started to implement device classes code. ...

Simple mocking interface now working.

Method signatures for Registry sketched out

Initial design for enhanced persistent registry done

Some ideas about device registration added

Method signatures for device classes added

Started to flesh out device classes proposed interfaces

Started to flesh out design thinking of device classes

Done a requirements and design pass of the device classes idea

Started a doc to spec and design the device classes

Python 3 now supported. ...

Logger now stores apparent power and real power headings.
Logger now stores temperature from eTRV in csv file

factored out python2/3 differences on raw_input

Updated device database comments for MIHO002 device

Now displays receive time of incoming messages

Merge pull request #55 from whaleygeek/unified_radio ...

Unified radio

Corrected incorrect comment in switch.py

Noted the layering of switching between tx/rx might be sub optimal

Merge branch 'unified_radio' of https://github.com/whaleygeek/pyenergenie into unified_radio

Fine tuned settings when sharing MiHome and legacy devices

Updated README

OOK(legacy) and FSK(MiHome) working in same combined.py build

Wrote a sample combined.py to use MiHome and legacy together. ...

Not yet tested

Renamed new radio2 module as radio, and removed old code

Removed any remaining spi-direct and tidied the build process. ...

Tested monitor.py, switch.py and legacy.py on Raspberry Pi.
All work from the same driver now.

Removed legacy direct-to-SPI interface. ...

This code now does FSK tx/rx and OOK tx from the same driver.

Merge branch 'unified_radio' of https://github.com/whaleygeek/pyenergenie into unified_radio

Started testing receiver

Tested FSK tx and rx on Pi, works fine ...

Tested both monitor.py and switch.py

Fixed compile typos. ...

Build test done on mac

Simplified the choice between len and cbp receive

Updated TODO list for next steps of radio.c

Implemented radio.c/radio_get_payload ...

Not yet tested

Implemented a better HRF_readfifo_burst. ...

This has facilities to read a fixed payload, or a count byte preceeded payload.

Framework for FSK receive in place

Started working through new FSK receiver architecture

Tested with MiHome adaptor plus, new FSK transmitter works fine

FSK Transmit coded, ready for test

Restructured switch.py so we have a tx-only test. ...

This is in preparation for pushing the FSK transmitter into radio.c

Refactored so that we can create a tx-only switch loop tester. ...

This will be useful for integrating a new FSK transmitter without receiver

Tested on Raspberry Pi, switch, monitor, and legacy work with spi scaffold

radio2.py offers temporary spi interface for radio.py ...

Tested on mac only.

Updated README to reflect recent advances

Merge branch 'master' of https://github.com/whaleygeek/pyenergenie ...

Test script updated

Build of radio_rpi.so updated

Legacy test passed

Defined manual release tests

Removed experimental arduino work

Fixed failed merge

Merge branch 'long-tx'

Tested with a house address captured from an Energenie hand controller. ...

Captured the bit pattern on GnuRadio and hand decoded.
Address was 0xA0170
Programmed this into legacy.py, and plugs previously learned
to the hand controller switched correctly

Tested with different house codes, works fine

Improved configurability of message repeat patterns. ...

Tested with different repeat lengths.
Optimum seems to be 8 inner, 1 outer transmits

Turned off TRACE on raspberry pi build

Tested with learning mode. ...

Works with all 4 switch patterns.
Can now independently switch 4 switches.
Some occasional RF reliability problems that need measuring.

Implemented mode switching logic. ...

This is so you can just say radio_standby()
Then radio_transmit() and it pops to standby at end of transmit.
Or, radio_receive()
Then radio_transmit() pops back to receive at the end.
Also massive tidy up and trimming out old comments.

Improved flow of code, specifically radio_transmit() ...

Pushes previous mode and restores at end of transmit.
This should auto-return to standby or receive as appropriate.
This will make it easier to implement mode hopping schemes later.

Simulation gets further now due to gpio_sim detection

Started design of new long payload transmitter

Initial design of new radio transmit scheme

Created links to issues for all items on the main TODO list

References to Energenie fixed

OpenHEMS is now OpenThings

Updated README with more up to date status update

Set repeats to get maximum OOK payload burst of 400mS

radio_rpi.so now built as part of the pi build process ...

legacy.py now uses the prebuilt radio_rpi.so

encoder now adds preamble. ...

lecacy.py now uses new C radio driver

Tested radio2.py on Raspberry Pi, now works with OOK transmit

Tiny tidy up

Renamed encoder_test to be consistent with other testers

Wrote radio2_test.py as a python OOK tester for new radio.so lib

radio2.py ctypes wrapper ready for testing

Preparations for new radio interface added to the architecture

Tweaks to radio_test.c layout

Tidied up radio_test.c readability

Removed unnecessary spurious radio configures

Fixed payloadlen bug that was introduced in API refactoring ...

Note: Reconfigures as OOK regularly, unneccesarily now

Fixed bug with OOK struct size in hrfm69_test.c

Finished coding radio_test and radio(tx) module

First pass of spec'ing radio.h/c and radio_test.c

Can now reliably turn energenie switches on and off with hrfm69_test.c

Coded a better payload transmit test routine for OOK

Started coding up new burst transmission scheme

Started to write another test case to blurt out OOK energenie messages. ...

This is only a test case.
But it has highlighted a whole load of discrepancies in the original C
code to do with payload length calculation, and the 'magic' zero byte
at the start of the buffer for stuffing the SPI register address in.

Tracked down the 'long sync bit' problem. ...

It's due to the HRF always preceeding each packed with an extra 01 at start.
Existing driver code sends 'chips' to prove this.
Looking like a preamble of zeros will be required to make this a non issue,
and then generate the real 80000000 preamble as part of the payload.

Got the radio chirping again after a long period of not working at all

hrfm69_test.c now sending a tone. ...

Discovered that 1/5 times it sticks waiting for TXREADY.
This might suggest an unresolved sequencing error in startup.

spi_test.c can now read back the version number of the HRF69 chip

Simple GPIO tester works on Raspberry Pi (flashes LEDs on radio)

Built a simple C test harness to start testing the radio with

Baud rate for OOK increased to 4800bps to bring in line with latest Energenie builds. ...

Sync pattern now recognised correctly as 0x80,0x00,0x00,0x00
Switching is eratic, but learning always learns.

Regular schedule for switch1 transmit test

Added feature to test a single switch repeatedly

Added green LED around transmit and receive transactions. ...

This is useful for measuring total on-air time on logic analyser

Experiments on Arduino with larger register writes and FIFO write ...

Getting some corruption on the Arduino RAM buffers.
Logic analyser traces show that the tables are getting corrupted
and the wrong data is being written to the radio registers.

Managed to write/read the mode register from real RFM69 with Arduino

Reset, select, write working on Arduino

SPI being driven on Arduino

Got simple 5 pin interface with GPIO working on Arduino Pro Micro

Preparations for wiring up to Arduino

A 'go for broke' OOK transmitter added to hrf69_test

Implemented most of hrf69.c write and read functions

HRF_write_reg written

Factored out all stdio from all testers and code

Mock read and write test working for radio register

Proved that we can read a register back from simulation

First pass at writing a test in hrf69_test.c

Marked the interface queries in HRF69.h

First attempt at specing out implementation of HRF69 based on python

spi_test working on Mac

Refactored delay into a platform module

gpio_sim working on Mac

Wrote GPIO simulator in a mostly platform independent way.

Framework in place for Arduino GPIO and simulated GPIO

GPIO is now tagged as RPi specific in build process

Created file architecture for arduino build

Started refactoring

Moved all C into a drv folder.

First pass at defining hrf69.h and radio.h interfaces

Started to refactor code into C

Pre-factoring work done prior to pushing more code into the C domain.

Tweaked payload format further. ...

Reset all devices and re-learnt the house code
Can now switch all devices on and off, together, and independently.
legacy.py test code now demonstrates this in a loop

Added a test mode to test all 16 patterns of legacy control messages. ...

Run legacy.py and try numbers 0 to F in learning mode.
In theory you should be able to select 4 different switch addresses
plus an 'all addresses', but I can't get this to work on my 7 plugs,
they all respond as switch number 1 regardless of what code I send
in learning mode.

Attempts to get multiple device addresses working. ...

Seems to be a discrepancy between documentation and sample C code.
Stopping here until I get this clarified.

Fixed legacy import errors

Renamed modulator as encoder

7 legacy adaptors now turning on and off with this trimmed code

Improved reliability of syncing of OOK messages

Got old code working again. ...

Looks like the extra bytes inserted by the buffer bug actually make it work.
This is probably related to sync bytes

Merge branch 'master' of https://github.com/whaleygeek/pyenergenie

Brought back old test code that was working

More experiments with payload timing

Corrected sync preamble. ...

Brought in original legacy C code for reference

Tweaks to legacy learn feature

Updated readme with latest status

Rewrote legacy.py to use new modulation code. ...

Not yet tested on a real Raspberry Pi

Refactored modulator into a standalone module

Modulator tested and working

Wrote first pass of OOK modulator and test harness. ...

Still needs to be verified against the actual test messages

Updated README

Slight improvements to timing of message transmission. ...

Still seem to get 'failed to send repeated payload to HRF' after a while
Suspect the FIFO is filling up or something.

legacy OOK test harness is working. ...

Note that sometimes we get 'Failed to send repeated payload to HRF'

Implemented a standalone switch exerciser demo. ...

Note this only really works with Adaptor Plus
(unless you seed the directory with your device id's first)

Started work on OOK support. ...

Ported C code into radio.py
Split out monitor and switch. Monitor.py is now just a monitor.
Switch.py needs more work to make it just a switch.
legacy.py will be a demo app for legacy (green button) devices.

Updated device database

Added rudimentary support for MIHO005 (HOUSE MONITOR) and MIHO013 (ETRV) ...

Fixed bug where eTRV sends an empty JOIN message that crashed the code
Added some extra fields to the CSV to support apparent power for House Monitor
Note: Voltage scaling on house monitor is incorrect.

Removed and tweaked a lot of trace messages to make it less noisy

Added missing file Timer.py

Updated README with present status and new planned items

Added a simple data logging to CSV capability

Added RESET at start to reset the radio to a known clean state. ...

Added a SWITCH ON/OFF message every 5 seconds.
Re-tested and reports now seem to be a reliable 10 seconds.

Improved message decoding. ...

Now builds a dictionary of seen devices.
Monitor FFFFFF messages removed, not needed.
Catches bad payloads and prints a warning rather than crashing.
Accepts and responds to a JOIN command.
Message and value encoders written and working.
Message and value decoders written and working.

Message decoder added. ...

Messages polled at a faster rate.
Now see unsolicited switch change messages from MiHome AdapterPlus.
Removed a lot of debug messages.
Real value decoding now working.

Improved encoding and decoding of OpenHEMS payloads. ...

Now uses pydict structures in the user app.
Encoding, encryption, decryption, and decoding done inside OpenHEMS.
This simplifies the application interface.

Corrected directory path

code formatting

formatting of code

Added source files from first working tests

Added some notes about sharing low level devices

Update README.md

Initial commit