I decided to change the project name from 1000.1000 to 1000² or 1000 squared.
Finally, enough pieces of the puzzle are starting to fit together so that I can have some end to end functionality: press a button here turn on a light there, while watching the actual slider change a graphical inteface. Oh, look the temperature is displayed as well. Hooray. Still need a few “cans of bug spray”, though.
First up, some nodes
Why are they called nodes anyway? Grabbing some old and new hardware prototypes, I made a basic system with a few types of nodes, what I thought was the minimum required to get started.
The node on the right most is the server or gateway node. Still using UART, hence the FT232 adaptor. It sends received packets to the raspberry pi in MQTTish format. It sends over the radio the received commands via UART, again in a MQTTish format.
On the bottom there is a simple remote node, with 4 buttons, which is battery powered.
On the left there is a PWM led driver connected to a lamp.
The left side one with the red antenna is a battery powered temperature, humidity and light sensor, of which I have a couple already.
On the top left there is a spy node, which dumps all traffic on the UART to get a check on the network. I have used this mostly on debugging.
On the top right is another LED driver, hooked up to my previously hacked lamp.
I am running things on a raspberry pi, but one could also re purpose some old computer.
I setup Mosquitto as the MQTT broker, to which Node-RED connects. In fact, I am using Node-RED as a link between the gateway node on the UART and Mosquitto, with everything else inside Node-RED interracting with the MQTT server.
Also built a small dashboard, here’s part of the hierarchy
Of course, one needs a very basic front panel, quite underpopulated at the moment. (there’s also a lesson here regarding halogen lights and temperature sensor placement).
Here are the nodes used to create the interface. With the exception of the function required to get time and date from epoch time, everything else is a normal node already included with Node-RED.
Monitoring the health of the nodes is also simple – since they transmit some parameters: the time since start (uptime), the battery voltage, the gateway RSSI for each and the average time spent in the loop. The number of packets received is also logged. Behind the scenes is very simple, since MQTT allows for wildcard selection of the same parameter from all topics.
Showing below is the time since startup of every node. For now they update this value every 10 minutes, but will change to longer once things are more stable. You can see a couple of nodes were reset some time.
A tiny bit of “automation” connects 2 buttons from the remote node to the 2 LEDs, sending the ‘toggle’ command so that each press turns the light on or off.
Back when I was deverloping the PSU burner, I wanted to have the Analog Discovery isolated from the common ground, to avoid noise and other issues. Since I did not have a way to do this, I ended up using a laptop on battery for measurements. But for long term, I needed to have this isolation. Unfortunately, things that can isolate USB at 480Mbps or faster are too expensive to justify.
The ADUM3160 isolator can provide a magnetically isolated 12 Mbps connection, which proved to be good enough. I grabbed one ready made isolator module from ebay for about $12, cheap enough. Well, it is not perfect: the B0505S DC/DC converter provided can only supply 1W and the Analog Discovery is a hungry beast.
Luckily, these DC/DC converters are cheap and I had some more in my parts bin. Their internal structure allows them to be connected in parallel, so I used 3, which should cover the 2.5W needed.
Next up, I had to upgrade the input fuse, to allow for higher current, my parts bin provided 0.65A one. Given the efficiency of the converters, this his slightly too small, so I would advice for a 0.9A one, the limit of USB 3 ports. Next up I have used an extra wire to connect the output of the DC/DC converter because the PCB trace was too small, dropping 0.3V on it already. Added an extra 470uF capacitor for decoupling, just to be sure.
The device works, but make sure to power it from a USB 3 or high current USB port, 0.5A classical is not enough. I am using a self powered USB hub.
I fired up Waveforms and set the signal generator to output a 1 MHz sine on a channel and used an oscilloscope channel to observe it. Luckily, the persistence mode does both a time count and a frame count, so I could use that as a method of USB performance. Without the isolator, I could do about 32 acquisitions per second.
With the isolator, the number dropped down to 9 per second. This is not so bad and using the thing without persistence in the normal oscilloscope only way does not show any perceived extra latency.
For about 15 EUR/USD and a couple of minutes of work you can make sure that your Analog Discovery is floating. There is some performance penalty, but there are few situations where this would matter.
As I was mentioning in my 1000.1000 Hardware selection, I have opted for the cool RFM69HW radio module. Weirdly enough, in quite a few sources (big distributor and ebay) the higher power HW module is cheaper. So there ie no reason not to get the higher power module, given quantity discounts. But I want it to operate at lower power most of times. The datasheet does not show any differences at lower power, so I had no reason not to go for the higher power module. It even says so on the features list on the front page, I can turn the power down to -18dBm.
Unaware of the full details, frustration emerged when I started mixing in both types of modules. The Radiohead implementation I am using (1.71) makes sure no single power value will work for both module types: over 13 dBm and a W module will not work, under 14 dBm and a HW module will not work. Digging through the code shows no problems, it does not really work as expected, but it should. The datasheet makes things more clear which power amplifier to enable based on the level and the library implements it properly. We can see there is some overlapping in the ranges as well.
A nice graph shows the power levels, from which we can see that the power ranges in the middle are achievable with different configurations.
So doing some digging around the datasheet of the IC, same one present on both modules, we learn that the IC contains 2 types of amplifiers. PA0 is the lower power one and is always directly connected to the LNA input and both to the antenna. For higher power, one has to use additionally the PA1 and PA2 and optionally a BOOST setting to squeeze out the last couple of dB. So how are these connected on the actual module. The RF69 datasheet shows a blurry picture of the module schematic for up to 13dBm output power, on page 76/77. We see that PA1 and PA2 never reach the antenna, just PA0.
Compared to that, the high power RFM69HW uses a switch which is driven by RX/TX pin, after some module reverse engineering. This means that only PA1 and PA2 reach the antenna, and never PA0, during transmission.
The fix lies in understanding how the modules work. Because it uses only PA0, the RFM69W can only have PA0 turned on and the output power ca be adjusted between -18 and +13 dBm. The high power RFM69HW can only use PA1 or PA2 or both, but never PA0, so the output power range is -2 to +20 dBm. Now the curves on the graph above make sense: the blue one works for RFM69W and the other colors are valid for RFM69HW!
This means that the code used to configure the modules needs to know which type I am using in order to configure the power output within the overlapping range. I fixed the function controlling the output power and will suggest the changes to the Radiohead developers.
With the power setting correctly implemented, the library can properly set a power level, within the limits, for either module. A quick test shows that knowing which module is attached, any power between -18 and +20 dBm can correctly set each module within the appropriate limits. Measuring the RSSI at a nearby node, we can observe that the RFM69HW is capped down below -2 dBm and the RFM69W is capped up above +13 dBm. The measurements were quick and dirty, so take them with a grain of salt.
There is a posibile hardware fix: the RF69 IC should make sure not to switch the antenna to PA_BOOST when the power level is set low, but that might require a new chip revision.
The software fix can get the RFM69HW down to -2 dB, with few users wanting to go lower as reducing the output power does not bring much benefit in power consumption.
Code fix for power setting function
Use for whatever other library driving the RFM69, original implementation coming from Radiohead library.
void RH_RF69::setTxPower(int8_t power)
_power = power;
if (_power < -18) _power = -18;
if (_power > 13) _power = 13; //limit for RFM69W
palevel = RH_RF69_PALEVEL_PA0ON | ((_power + 18) & RH_RF69_PALEVEL_OUTPUTPOWER);
if (_power < -2) _power = -2; //RFM69HW only works down to -2.
if (_power <= 13)
// -2dBm to +13dBm
//Need PA1 exclusivelly on RFM69HW
palevel = RH_RF69_PALEVEL_PA1ON | ((_power + 18) & RH_RF69_PALEVEL_OUTPUTPOWER);
else if (_power >= 18)
// +18dBm to +20dBm
// Need PA1+PA2
// Also need PA boost settings change when tx is turned on and off, see setModeTx()
palevel = RH_RF69_PALEVEL_PA1ON | RH_RF69_PALEVEL_PA2ON | ((_power + 11) & RH_RF69_PALEVEL_OUTPUTPOWER);
// +14dBm to +17dBm
// Need PA1+PA2
palevel = RH_RF69_PALEVEL_PA1ON | RH_RF69_PALEVEL_PA2ON | ((_power + 14) & RH_RF69_PALEVEL_OUTPUTPOWER);
I got a Peaktech 6225A power supply to power some things, as it seemed like a good deal, going beyond what one might find normally in these types of supplies: more display resolution and supposedly, lower noise. For this price, this supply is a good deal compared to other similar ones on the market. Let’s see how it performs.
You can even grab one on Amazon.de
through affiliate links which helps me support this website, while offering you the same low price.
Voltage and current set
For the first test, I have checked if the supply is really delivering on that 1mA / 10mV precision, using a calibrated multimeter. The results are below: for the voltage setting, the power supply seems to hold on pretty well from about 0.2V and higher, below which it is not that much control. On the current side, things are similar, current cannot be very accurately set below about 50mA. In both cases, the precision starts to be there from about 1% of the full scale and I think it is perfectly fine for this kind of product.
First off, the power consumption test: the power supply burns 4.2W while on and with the output turned OFF. If the output is turned ON and set to 25V and no load, it consumes 5.2W, which I think is decent.
Of course, I had to grab my PSU burner to start stressing this guy out. I set the supply for 30V and 5A load and let it run for half an hour. The fan quickly started.
Mains power ON test
I checked what happens when the power supply is switched ON, from the mains switch at the back. With no load, there is a small 0.5V glitch that takes time to die down, due to the high filter capacitors. With a 10R resistor the glitch is surprisingly higher at 1V, but it extinguishes faster. Note: voltage is orange channel and current is blue channel.
With 10R load
Soft power ON test
Setting the power supply to 5V and turning ON results in this waveform. Note that the oscilloscope is suffering from external noise due to grounding.
And powering down, again no load shows quite a long time to discharge the capacitors
Here is the response when the supply is set to 5V and a 1A load pulse is applied. We can see the supply goes back on track within about 100µs. Note the voltage drop with load is not that high, but the Analog Discovery measures it after some wires. Disconnecting the load produces about 1V higher peak which is not that good.
Constant current mode
This is one of the interesting thing to check, as it shows how the supply will perform in case of a circuit fault and limit the current. First test, 5V output with 1oomA limit, checking what happens if the PSU burner tries to pull 1A from the supply. It takes about 6-8ms before the current is limited to the set value! This is quite slow.
Next up, same test, but setting the supply to a 1A limit and trying to pull 2A from it, we see the same response time.
With the same 1A limit trying to pull 5A shows some more details: the response time is now shorter, about 4ms, indicating this is closer to the loop response, and not the discharging of the output capacitors. The oscillations also show the loop stabilization time.
This is further seen as what happens when starting the power supply shorted, but with a 1A limit, which allows the load to pull almost 4A at start.
Conclusion: the supply is quite slow and will allow a pretty high current to pass through a shorted load before the loop reacts, not recommended as protection to delicate circuits.
Finally, let’s look at noise test, this time I removed the whole PSU burner and left the Analog Discovery only. Setting the supply to 5V and loading with a 10R resistor, I checked the noise.
The noise is about 80-100 mVpp, which through a slightly incorrect way to judge this, comes down to about 15mVrms, quite more than the 3mVrms quoted in the specifications.
Actually, manufacturers don’t like to measure noise like this, but by having a 20uF capacitor at the end of the load. So doing the same thing, I get about 12Vpp of noise, much better and quite in line with the specifications, which is really pleasing.
Of course the fun does not stop here, so let’s look at a what is inside. Surprise surprise, there’s a big TIP3055 transistor on the bottom, more on that later.
In general we see things that are expected: the input socket contains a fuse and the 110/220V switch, with the power then going through a double switch. Furthermore there is some common mode filtering and your typical NTC to protect from high power up rush in current.
We can also observe the thermostat of the fan is bolted on the middle heat sink which cools the main switching transistor. My non contat thermometer shows the fan turns on at about 35-40°C.
Comparing to other types of these supplies, like the Manson NSP 3630 (a lot of the designs are copies of this more or less) the main board, visible in the middle, appears much simpler.
Actually, the design is rather different. On top there is a regular 50Hz transformer, with 3 secondaries, compared with an auxiliary switching supply found on the Manson. On the bottom, the blue transformer is the main power supply transformer, carrying out the load.
The supply is controlled by an OB2269 switch mode controller, not the typical TL494. The OB2269 has quite a few advantages, it’s designed to drive a MOS which can be more efficient and it is optimised for “Extended Burst Mode Control For Improved Efficiency and Minimum Standby Power Design” which makes it more suitable for this type of application.
Unfortunately, as expected at this price range, the “Chong” filter capacitors are not the strongest point of this device.
The output filter of the main switching supply is a pair of probably low ESR 1000uF/50V 105C SME capacitors. Can you spot anything missing? Yup, no inductor filter on the output.
Looking at the board from behind, we can see some clear demarcation of the high and low voltage regions, which appear to be safely separated. Also here you see an element which is out of place: the wires of the fan are soldered directly on the back, even though there is a connector in place there. While the supply is quite low cost, there don’t seem to be any other elements inside aimed at saving such a tiny amount.
Next to the clandestine fan connection you can see the 3 rectifier bridges generating the 3 auxiliary voltages used for the supply control. The fan is supplied by 10VDC and there is another 30VDC supply and a -6VDC negative supply, all going to the front panel.
Moving on to the front panel, it connects with a ribbon cable carrying those 3 supplies plus another wire going to the base of the TIP3055 transistor on the bottom. There are 3 more thick wires connecting to the banana connectors on the front panel.
Here is an overview of the whole front panel, first the front side. We can see the two 4 digit, multiplexed displays, the 3 LEDs, 2 rotary encoders and the soft ON/OFF button. Across the outputs there is a 100nF capacitor, plus 2 more from each pole to the earth. There is a reverse protection diode as well and the bottom left shows the current shunt.
Moving towards the back now. Voltage seems to be regulated by a couple of TL431 (top, middle) and transistors. The big decoupling capacitors are SME 470uF/50V, 105C each and they are directly in parallel to the output, so the current measured in the shunt will be delayed by the discharge of the capacitors.
The design is much better than the Manson and other supplies, which relied on a lousy ATMEGA micro controller generating quite low frequency PWM signals which are then filtered to generate the voltage references. The readout was then carried out with dedicated voltmeter ICs.
Here we see a more modern approach: there is a micro controller handling everything, which of course has the markup removed.
Starting on the top, the display is controlled with two 74595 shift registers, one drives the segments and the other the digits, through the transistors on the right side. I have not poked around as to why there are only 6 of them and not 8, one for each digit. The buzzer and driving circuits are not populated.
On the bottom side we can see the micro controller and the control loop. Based on the capabilities of the supply, the microcontroller is probably a more modern device featuring 12 bit ADC and maybe a 12 bit DAC. Given the 3.4V supply I measured, I suspect filtered PWM is not involved here. Something like an XMEGA32E5 could do it, but the pins don’t match. The microcontroller is a 32 pin LQFP with 0.8mm pitch, power seems to come from pins 7 (GND) and 8 (VCC), and pin 1 might be reset. Any ideas?
The loop is controlled by 2 OP07C opamps, which feature very low offset (60µV typical) as a requirement to keep things under such high precision. Unfortunately the op-amps are also quite slow, with a unity gain bandwidth of only 0.4MHz, which can be seen as well in the loop reaction time, in the tests above. There is another 4558 opamp which I believe checks if the supply is in CC or CV mode.
Going back to the full view, the opamp near the connector is a good old 741, which I believe drives the transistor next to the connector which in turn drives the big TIP3055 transistor in the supply.
Speaking of which, what exactly is the big TIP3055 transistor for? Well, it is actually used as a linear regulator following the switch mode supply, which is a feature this supply has and not the older designs. I have measured and it keeps about 2.3 – 2.5V across collector-emitter, depending on load current, which means we are looking at about 12.5W of dissipation at max load. This secondary linear regulator allows the supply to provide such low output noise compared to others.
The way it works is that the opamps on the front panel drive this output transistor to the required output voltage. The switch mode supply follows suite and is designed to supply about 2.5V more than the output voltage, as seen by about 2.5V CE on the transistor. This design provides the best of both worlds, the switch-mode supply gives higher efficiency in a lower size and weight, while the following linear regulator can filter out most noise and provide a cleaner output.
I tried poking around the two un-populated headers with both the Analog Discovery digital IOs, a USB-serial adapter and a ST-LINK V2 and could not get any response out of the micro-controller in any way. I don’t intent to spend more time reverse engineering this, since it needs to go to work.
The power supply delivers great results for the price compared to other similar products, I believe it is a good deal to have one of these for general purpose, with the exception of very delicate things. Furthermore, the dual switching and linear regulator achieves a low weight and compact device with low noise, as promised.
However, the supply is not without it’s limits: the high output capacitance and rather slow loop means it is capable of delivering quite some punch before the limit sets in, so not everything is safe to be powered from it. Do note however than linear or not, only much more expensive lab power supplies have a fast acting loop with little output capacitance, as such a feature costs and is infrequently required. Quite a few other small improvements could make it better: screw type bananas are better for connecting wires directly and some sort of proportional to temperature fan control is quieter and welcome in places like a home lab.
Finally, I do recommend this for a well made and great performance for the price this product offers. You can even grab one on Amazon.de
through affiliate links which helps me support this website, while offering you the same low price.
What does one do when designing a power supply? Well, build a power supply tester, of course. One of the simplest things to build is a constant current load. This will allow for testing of the endurance of the power supply, as most of the designs out there are using slow components.
However, I wanted to make a better one: one that I could hook up to my Analog Discovery and generate a test waveform to be able to connect and disconnect the load fast. This is a weekend project, so all parts are not the best for the purpose, just what I had around.
Warning: The analog discovery is grounded through your PC (unless using a laptop on battery not) so you have to make sure that both the supply being tested and the auxiliary 12V supply are isolated and not grounded. Or use an isolated aux supply and a USB optical isolator if the supply under test is grounded.
The simplest of design, one OP-AMP driving 3 MOSFETs. All fine until I made a mistake: with the signal generator connected and turned ON, i connected a 24V 5A supply that was already on, while the signal generator was providing a square wave for 5A current. This caused one of the MOSFETs to blow up (shorting) due to an avalanche event. Finally the power supply went into hiccup mode.
Have you ever seen a fuse pulsating? With the output of the supply shorted and the fuse quite at the limit of the current, it is almost melting each time the power supply tries to restart
There are some improvements, so let’s go through the schematic first. The way it works is simple: U1A will drive Q1 such that the voltage across R6 is equal to the reference at it’s positive input. This in effect creates a constant current being drawn from the test power supply.
In “external mode” the constant current load is controlled and read out by the Analog Discovery. Since the control signal can be up to 5V, with R5 being 0.1Ω, I have divided it 10 times using R2-R3, to limit the max current to 5A. In “internal mode” there is a fixed reference generated using RV1 and a 5V regulator, which makes the supply draw a constant, adjustable current.
For a nicer look, I added 2 meters, they are your typical 3 wire voltmeters, but make sure you get the nicer ones which auto-scale and use all 3 digits for precision under 10V which are also hackable. The current meter needs a 10X amplifier first, since we are reading current on a 0.1Ω resistor.
Finally, let’s look at the extra safety features: first, I am now using 5 transistors instead of 3, of similar type. I have added a rather large, 0.5Ω balancing resistor in each source, made by 2 1Ω/0.6W resistors in parallel (remember, built out of what I had in my component drawers). And second, a Zenner diode between the source and drain. This will serve 3 purposes: if the load is applied with reverse polarity, the fuse will blow and it will also protect the transistors from high peaks when a load is disconnected. Third, if more than 36V is applied, it will blow the fuse. The nice thing about using a Zenner like this: if a high enough current is passed through, it will melt and become permanently a short: it forms an anti fuse.
Oh, and did I mention I want it to be fast? Well, that is why I used a TL3472 (besides that I had one): it has a rather high slew rate and a high output current capability.
Transistors tanning. Notice how this heatsink has been through a lot…lots of holes.
First tests, including cooling:
Preparing to put things together:
First test. The front panel is 3D printed. Also the knob, I did not have one for perfectly cylindrical shaft multi-turn potentiometer.
Testing the power brick guilty of burning the previous transistors. After a long time, the heatsink reached about 45°C in the middle at the well cooled area and 53°C at the edge fins where there is not much air flow.
And finally testing the speed using a 7805 regulator and short output wires using a 10kHz, 1A load. The rise and fall time are about 2.1µs.
<disclaimer> This article is an emotional rant and may contain some strong language.</disclaimer>
Studying the available open source solutions for my 1000.1000 project, I found quite a few interesting frameworks for DIY sensors. One of the most complete and fast growing is the My Sensors project. The Framework, like many others, is based on Arduino, with both the traditional and more exotic hardware.
I did quite some embedded software development and other programming throughout my life, as some projects on my site show, but only from a classical approach. I have used multiple Arduino boards for low price and convenience and I have ported a few Arduino libraries to a regular environment, but my experience with the pure Arduino way of software was very limited. I was aware of the limitations, but not quite so…
Playing with examples
Getting the MySensors to work was fast, I setup a gateway, a dimmer node and a button node. Grabbed openHAB controller and made a rule to toggle the LED at the press of the button. Added a motion sensor and another rule in openHAB to turn the light on for some time when there is motion. Sweet. Great work mysensors.org!
Ok this looks really easy and well off, let’s dig into how this works. Oh….s**t! Can’t navigate through the code using the Arduino Ide, well, ok, fine, I knew it was basic.
Let’s just open the source files of the MySensors library used in the example and look at them. S**t! The fu***ng Arduino IDE cannot even open the very .cpp and .h source files used in the libraries without some inconvenient hacks.
OK, there are other IDEs out there, surely Atmel studio has been bragging to me for a while that it supports Arduino. Fire it up, open the project, and voila! Can jump to function definitions….but wait! All the files used in the project are not easily visible and it is terribly difficult to figure it out the whole mysensors library. Better, but not good enough.
Let’s convert the project to a regular Atmel Studio project.
Much better, all most of the source files required are pulled from the Arduino folders into a nice, complete project where one can see all the things happening in the background of the Arduino. And yeah, Arduino does a lot of things behind you back and uses some hardware blocks inside the microcontroller, but that is another story.
But it won’t compile. See, for the sake of simplifying things, Arduino code breaks some rules that normal C/C++ expects. While the Arduino processes the files in the background to make sure they become compatible with gcc, this does not happen here anymore since this project is now a normal C++ project. Untangling things would take a significant effort for such a complex library.
As far as I know, there is no easy way to get both things – the nice complete project with all sources and advanced IDE features and Arduino to compile it, but please correct me if this exists. This is valid for other IDE alternatives, like Eclipse and it is caused by the Arduino way of life.
How does it feel?
Arduino is supposed to be like some aid to swiming. Only that it turns the waters muddy so you cannot see what’s below and you cannot actually swim, you are forced to walk through water. It sorts of works if the water isn’t too deep and your purpose is to get across the river, not learn how to swim.
Don’t get me wrong, it has a very useful place in this world. The community around it had built one of the largest collections of open source embedded code and the far east manufacturers can sure get a board in your hands for cheap. Who cares if it uses a counterfeit or out of spec chip or both, you can get a lot of stuff done.
The alternative is difficult
Actually it isn’t. Let’s look at an example: try reading an analog voltage.
This is how the Arduino sketch looks like, independend to what Arduino hardware you use. Let’s try the newer Arduino Zero.
There is some setup, but nothing to do, it happens in the background, and then you just loop through reading the value. Looking at the ADC it feels like there is more to it, but changing settings will get you digging around the internet a lot.
Now let’s take a similar example using a the SAMD21 ARM that is found on the newer Arduino Zero but in Atmel Studio, which is at the moment one of my favourite microcontrollers. The datasheet shows that indeed, ADC does have few more tricks up its sleeve:
Atmel studio has built in example project to read the ADC using the Atmel Software Framework (ASF). This is the program it produces (removed comments):
Here’s what you immediately learn by looking at the code: there’s a main() function where everything happens, which Arduino normally hides. system_init() takes care of some basic initialisation of the chip. Then you need to initialize the ADC. To get a value you need to tell it to start converting and wait for a result while the ADC is busy, because this takes time. A little bit more complex in appearance, but no different than what actually happens if you use an Arduino, remember the hardware is the same.
But here there is nothing hidden, nothing happens out of your sight or control, you have now discovered more is needed to make a microcontroller work. You can explore the whole code with the ease of a powerful IDE. And here is where something magic happens, have a look at what is inside the ASF provided function that sets the default way to use the ADC adc_get_config_defaults(&config_adc):
It immediately becomes obvious what are the different settings one can play with. Even for a total beginner, jumping back and forward between this configuration and the ADC diagram above maked it pretty obvious how things can be changed to do a lot more if the user wants to. You don’t need to understand everything at once, a lot of things will be intuitive and some will need the datasheet. This is the way embedded programming normally works, you control the hardware, not use in whatever hidden and obscure way the Arduino wants you to. Atmel provides this nice hardware abstraction layer through ASF, but one can, of course, also use the most efficient way of editing the actual configuration registers following the datasheet.
The complexity is always there, but in Arduino it is hidden. This might be good, but it is also inaccessible and that is why it remains a mystery for a lot of users who never grow beyond it. It’s exactly the wall you hit when you want to dig in and see what really happens under the hood that stops you.
My advice: use Arduino for simple fast things, profit from the community written libraries and buy the cheap hardware, but do learn about the whole world beyond it.
I am not complaining about the loss of performance due to poor code implementation, and in the end not even about the bad IDE. It’s the hide things under the rug to make it look simpler and cleaner approach that upsets me.
So where does this leave me with my home automation project? Tough dilemma, I like my ARM with great IDE and partially written code, but a readymade platform covering 80% of my needs based on cheap hardware and terrible Arduino is also tempting.
Note: there are no links to Arduino sites because they split up some time ago and I don’t find either of them more relevant than the other or worth including both.
[Under construction. Check out The intro in the meantime]
There is plenty of software to use in home automation, but of course I decided to stay with open source and free options.
In recent years there have been multiple new “controller” type of software, which is the type that is suppose to control, manage, display, graph, talk etc while working with products from multiple manufacturers. This gives the user a nice central interface to control and display, coupled, of course with automation rules and scheduling.
Node Red is a cool new idea, coming from IBM, implementing flow based design. Multiple people use it for home automation and just like Open HAB it has a plethora of ready made possibilities to interface with existing products. It can run quite smoothly on a raspberry Pi offering a web interface for designing the flow. It provides ways to build a dashboard through simple hierarchies and predefined widgets, although some things are more limited than I would like.
At this moment is seems like the best support to try crazy stuff, like making the umbrella blink if it is going to rain tomorrow.
MQTT is something that is not really needed, using either Node Red or Open HAB messages can have any kind of format. But I decided to use it because it provides structure and can interface different software components, as plenty have built in support. The principle of MQTT is simple: there’s a central broker through which all messages go through. A message is made of a topic, which is hierarchical, (like house/first_floor/bathroom/temperature) and a payload containing the value. I find the hierarchical naming a little bit against human nature but it does provide the benefit of easily selecting similar things through wildcards.
[Under construction. Check out The intro in the meantime]
A picture says 1000 words!*
*inner voice dependent
When I began this project, I was long given up on the ATMega and ATTtiny micro, with some exceptions, as they are not really justified in a hobby environment. Therefore, I have initially intended to make my nodes using an ATXMEGA32E5, which I believe is an ideal device for sensor nodes, due to the various and low power peripherals. However, it soon turned out that for some devices I would need more flash, but the family stops at 32K and a package change is required for more. So I dropped it in favour of the new SAMD21, ARM cortex M0, which comes with more flexible package and flash options. This device burns a little bit more in sleep, but still tolerable. And a 128K flash and 16K RAM version costs slightly more than the lousy old atmega328 with tons of extra features (way better ADC, DAC, more PWM, more timers, more I/O, more flash, more RAM, faster core, lower uA/MHz, RTC, more SPI, mor UART, more I2C….i’ll stop here).
Therefore, welcome the “core 1” SAMD21 node and IOT board.
The top and bottom headers contain all the pins of the microcontroller, including power. The left side has an easy breakable micro USB and room for the antenna. On the right there is the “basics” header which contains power plus 7 other carefully chosen pins. For most nodes, the right header will cover all the connection needs. An antiparallel red-green led can use 2 pins and no series resistor, while being able to display 3 colors. As mentioned in the lessons learned, there is a linear regulator, for the projects that need a regulated 3.3V supply.
And the back featuring the RFM69 radio:
The best way to see the power of this little beast is to look at one of the earlier test modules i built. At the same time, I was using : I2C, light sensor, ADC channel, battery monitor, motion sensor, contact, button and driving WS2812 and 3 PWM channels, red-green status LED and, of course, the radio. It’s not that a lower end microcontroller could not do it, but the flexibility and free memory I still have while doing this is of very high value. And, although I am not proud of it, I took the non elegant solution of driving the WS2812 LEDs by bit banging, because I totally can with a 48MHz clock.
The radio used to transmit the data between nodes is one of the crucial parts of the system. As mentioned in the Lessons learned, the NRF was a good candidate, but eventually failed to meet the needs, so I chose the RFM69. Truth be told, I believe the perfect radio is a combination between the two and some bonuses.
Here’s what I would like to have:
1. Integrated PCB antenna is generally better than an external one when designing a compact sensor node.
2. Decent power, 100mW or more, with the possibility to scale it down to 1mW.
3. High range of bit rates: the RFM69 can do 250:1 (300kbps to 1.2kbps) while the RFM75 is only 8:1. Lowering the bit rate allows for higher range when more power is not an option.
4. Automatic packet handling with automatic ACK and re transmission is gorgeous.
5. Automatic bit rate detection from the preamble: this would allow a gateway or any node to receive any packet without prior knowledge of what bit rate to expect.
6. Cost should be somewhere between the RFM75 and RFM69 (1 to 3 EUR/USD each).
7. Sub GHz band has quite some advantages, but for compactness 2.4GHz works too. Same module with integrated PCB antenna working at 433MHz or 868MHz or 2.4GHz would be gorgeous.
This page covers things I have learned and why I did what I did in my 1000² project. Some experience, random calculations, opinions and thoughts.
So what is the battery philosophy? Should I make everything last as long as possible and change batteries each time some device is dead? What if I have 100 devices at home? Should I use the smallest battery that lasts 1 year and change all of them every year? The expected lifetime for motion sensors will be close to 1 year of life while devices like temp sensors could go for up to 10 years on AA batteries, with others falling in between.
I am staying out of energy harvesting, be it simple solar or more complicated as this is not a universal solution. Long operating times are a must, randomly needing to replace the batteries of many devices every few months becomes quite a big hassle, even for a small home.
My battery philosophy is simple: change the batteries only once a year for the required devices. So, everything should last for a year, guaranteed, worst case scenario. For very long life devices, the ‘low battery’ warning should allow 1 year of operation. So, a yearly battery change should be comfortable enough.
On top of that, no weird shapes, just CR2032, AAA and AA.
Inside the devices, the brains will be a microcontroller, while radio will take care of communication, we’ll call this ‘the core’. Finding the optimal pair is not an easy task. There will be a huge diversity of devices and tasks and one size fits all might not be the best way to go. At the opposite side of the spectrum, the optimum size for each function will blow up in an unmanageable diversity. I know how little it takes to get a button and an LED on the internet, a few times more than that to measure a bunch of LEDs and quite a lot more to get a nice user interface. I am aiming for 2 flavours of the core: devices that need to be very power efficient and devices that can feast from an all you can drink buffet of electrons. If possible for the 2 aromas to merge into one and get power efficiency and performance, even better.
Universal – particular
Going full universal and making a node board suited for all the required tasks is complicated. Let’s just look at the power supply: some devices can work from most range of 2x AA batteries, some need at least 3V, some will need more. Combine with USB power when connected while sipping every last bit from batteries and it gets complicated to manage already.
In my experience, using small scale assembly, getting 40 simpler boards (20 & 20) designed and assembled is simpler than designing a more universal one and then assembling it with missing components and different configurations.
I counted about 100 simple devices that I want to measure, control or interact with in my home for now. A simple device is a switch, a light, a motion sensor, temperature sensor, button, led, small display, power meter, switch, plant humidity etc. The combinations of these will result in functionality of the system, like a motion activated light. In order to minimize the amount of hardware, I have grouped them by functionality and location. Keeping the universal – particular balance, I grouped the devices in “common denominators”. The surprise here was that each core needs to handle just 2 devices on average.
This points out to the complexity required for each ‘core’. Drop by the Hardwarepage to check the solutions.
Software and hardware modularity go hand in hand. While OOP is not my favourite on microcontrollers, it’s the best way here. Fundamentally, most devices will fall in just a few classes (GPIO, analog I/O, read/write at a specific I2C address, PWM, counter etc). Allow these types of classes to operate on any hardware resource and the platform becomes highly modular.
Step up or step down
A step up regulator used for this application like the TLV61225 does not squeeze every last bit of battery when using a power hungry radio. The radio I will use needs 45mA while transmitting. Supplying this current from a step up regulator providing 3.3V from a pair of discharged AA batteries at 2V proved problematic, as this requires close to 100mA of current, taking into account the efficiency. Because of the high internal resistance, the voltage drops too much shutting down the node with significant charge left in the batteries.
The other alternative, of using an LDO from 3 AA(A) batteries provides a longer battery life as it discharges them at half the current (taking into account the different number of batteries). Efficiency is not that bad either, as with 3.3V out and a fresh set of batteries (4.5V) it starts at 73%. As the batteries discharge, the efficiency increases (funny, ha).
How much does it cost to keep an ESP8266 running per year?
Assuming always on, 70mA from a standard 5V supply through linear regulator, 0.35W consumed. Adding inefficiency and self consumption of DC/DC converter getting to 0.6W from mains(I measured). This is 5KWh per year, or about 1.1 euro at average EU price. If the price does not tell you anything, think of the effort for a human to produce this: your not so sporty, but decently conditioned person would spend 50 hours on a bike to generate this electricity.
A pair of AA batteries will set me 0.4€ (in some quantity) and have the potential to keep a low power radio running a few years. So a low power radio battery powered temperature sensor will cost more in the long run compared to a cheap, mains powered ESP8266 temperature sensor.
NRF24L01 goes a long way….or does it?
I have used a hell lot of NRFs in my project, Stockie and encountered no major issues. They are super cheap and easy to get…but the very low transmission power limits the range. Not so fast! By combining the low power modules on the Stockies with a high power modules and antenna on the gateway I could get a lot of range (+/- 1 floors and anywhere inside the apartment). With a caveat, that did not matter for the application: allowing a lot of retries, just to play it safe.
I started working on this project with a RFM75 in mind, which is a clone of the NRF as well. It comes from a distributor, because of my ‘no cheap china‘ rule. The first lesson: automatic ACK does not work between cloned NRF modules and RFM75 modules. Rumor has it that Nordic hid some things in the datasheet and some bits are actually flipped in the packet, with the NRF clones following the datasheet while the RFM following the over the air transmission. Hence, automatic packet ACK and retry does not work between the sensor nodes (RFM75) and a NRF24L01 PA LNA (=NRF clone) gateway.
Implementing ACK in software (I used the Radiohead library) showed a higher on air time for the modules, hence more power. This really shows the advantages of the auto ACK the module does in HW.
Moreover, modules located far from the gateway required a significant amount of retries before getting a packet through, quite frequently. This leads to higher battery consumption and unpleasant delays, which could end up being a second even, which pretty much killed this module. I have tried the system with a single node, so only the nearby WiFis as pollution, though quite a lot of them.
So I moved on to my next favourite, the RFM69, which is superior due to operating in sub GHz band, more power and lower data rate. Check out the hardware section for more thoughts on this and my wishes for the perfect radio.
Bad Watchdog, Bad!
As a way to increase that chance that everything works and because sometimes my RFM69 modules were hanging (similar to this, still don’t know why), I started using the watchdog timer (WDT). Courtesy of the complex and flexible clock generation of the SAMD21, the WDT is fed with a 32 Hz clock, which allows for a max timeout of 8-9 minutes. This covers my longest sleep time of 5 minutes.
Soon after, I realized that my light dimmers were behaving weird when transitioning – something that is controlled within the infinite loop, which normally needs 3-4 ms to pass through everything. Cue some debugging and it turns out the WDT required over 50 ms to reset. Why? Because ARM micros can have multiple clock domains, which have synchronisation registers between them. Feed one of those domains with a slow 32 Hz clock and that is why it takes 50 ms to reset the WDT. So I made the compromise of increasing the clock and reducing the WDT to 2 minutes and limit the sleep time to 1 minute. This has shorten the reset time, at the expense of not allowing some nodes to sleep more.
Sleepy nodes will need about 30 ms to reset every wake cycle (default sleep time is 1 minute), which combined with about 2.5mA for the CPU current results in an average current of 1.25uA, little but not insignificant. For the awake nodes, I reset them every 100 ms, and simply don’t wait for synchronisation of the registers (it happens in the background).
Note that a similar situation can happen with the RTC, requiring a long read time, but that is more flexible because the RTC has an internal prescaler as well, allowing for a higher input clock.
When dealing with battery powered radio nodes, it’s naturally desired to send as little information as possible, each extra bit requires energy. Therefore, when a node sends it’s data, the minimum needed is an identification and the value, provided that the receiving end knows what type of device is the one with the specific identification. Given the size of the network, I was considering allocating a byte for the node address (the hardware), another byte for the channel number (so I can have an identification for each channel, eg voltage 1, 2 3) and another byte for the type of channel (eg voltage). Finally, the value is send in whatever way it is stored (float, integer, string) and the gateway needs to know how to decode it.
But, there is a catch: having predefined types means restrictions and having to configure all those predefined types in multiple places. It is one of the reasons why I stayed away from MySensors: they use predefined types and adding one requires updating the whole library.
My solution was to replace a channel number and type with simply sending channel_name=value in text format over the radio. This gives me some advantages: first, the originating node has total control over how the data is displayed (it is normal to have 3 digits for an 8 bit ADC value or 5 for a 16 bit ADC, even though you will always send a float number). Second: there is no middle software that needs updating to know when a new type of data arrives. Third: with a self explanatory channel name, it will be very easy to know what to do with the new sensor that pops up.
Ok, so what’s the damage, I am obviously wasting more power? A typical transmission looks like this: once a button is pressed, the node wakes up. About 2 ms are used (-2 to 0) to read the data and process other nodes plus preparing the radio packet, then the big current spike is the transmission then the longer time (~6 ms to ~30 ms) is waiting for ACK, with the radio receiver ON.
Below is the radio packet format that I am using. Taking into account the overhead required for the RFM69 module and the Radiohead library, 14 bytes on top of the actual payload are required.
Considering the time & current for transmission, reception and for sleeping, the energy consumption for different lengths of payload (2, 10 and 20 bytes) can be seen below:
When all is taken into account, an average length of a text payload of 10 bytes requires just 16% more power than the ultra short 2 bytes one.
At the extreme end, sending 20 bytes requires 35% more than sending 2 bytes. This is great, the 16% or even 35% more power is something easy to manage: you will not care that your temperature sensor updated the data 16% less frequent to compensate of it. It’s manageable and saves a lot of complication.
(of course, in a distant future, a node could negotiate with the gateway automatically to send this more compressed, but in the meantime it is not a deal breaker).
The ceiling lights above above my desk are halogen spotlights, which I used to take some pictures. Moving the temperature sensor node for the picture I later noticed almost 4°C in temperature increase. A quick check with a shade confirmed – the halogen lights heat the black SI7021 temperature sensor by about 4 degrees. Or could it be the exposed die is sensitive to light?
I have enjoyed building small home automation devices for many years, simple things that make life better or just cater for the lazy animal. The world has been dreaming about the internet of things where everything is connected for a while now. While the big guys are focusing on intelligent AI assistants, the market for simple devices is still not mature enough, being full of overpriced products that don’t want to talk to each other.
This leaves plenty room for diy projects, which at this point in time can be put together easily thanks to the dropping cost of electronics coupled with multiple open source software packages.
A bit of history
I started to automate things using the traditional computer parallel port and a literal bucket full of relays which drove my thirst to more complicated things that could operate independently. I jumped on to microcontroller projects and a few years later I made a small RS485 network. Moving taught me the important lesson of breaking free of the wires: some devices simply need to be wireless, while the plethora of others could go either way, saving the user from the ugly and complicated cabling experience. This is not an easy job, though.
Is my new ambitious project: do a lot of home automation while spending 1000 euro and 1000 hours working on it. Well, hopefully not that long time, but a budget and time limit give a good motivation to get things going. Plus, both constraints avoid the hobby trap of infinitely fiddling with details.
Some ground rules.
The budget limit is for final parts kept in the system. Failures don’t count, but are great way to learn.
No cheap china: parts used should come from good sources and be of consistent quality, as much as possible.
Distributed intelligence and context aware devices.
Cut corners, where possible.
Attention to security.
Modular hardware: easier to design and debug.
Modular software: easily build a new thing based on old ones.
Separate by function, easy to reconfigure.
Anything that runs on batteries should last at least 1 year in worst case.
Allow room for growth.
More to come….
Time to get moving. Make sure to check the related pages .