Tag Archives: hardware

Statistics and Hacking: A Stout Little Distribution

via hardware – Hackaday

Previously, we discussed how to apply the most basic hypothesis test: the z-test. It requires a relatively large sample size, and might be appreciated less by hackers searching for truth on a tight budget of time and money.

As an alternative, we briefly mentioned the t-test. The basic procedure still applies: form hypotheses, sample data, check your assumptions, and perform the test. This time though, we’ll run the test with real data from IoT sensors, and programmatically rather than by hand.

The most important difference between the z-test and the t-test is that the t-test uses a different probability distribution. It is called the ‘t-distribution’, and is similar in principle to the normal distribution used by the z-test, but was developed by studying the properties of small sample sizes. The precise shape of the distribution depends on your sample size.

The t distribution with different sample sizes, compared to the normal distribution (Hackaday yellow). Source: Wikipedia

In our previous example, we only dealt with the situation where we want to compare a sample with a constant value – whether a batch of resistors were the value they were supposed to be. In fact there are three common situations:

  1. You want to compare a sample to a fixed value: One sample t-test
  2. You want to compare two independent samples: Two sample t-test
  3. You have two measurements taken from each sample (e.g. treatment and control) and are interested in the difference: Paired t-test

The difference mainly affects how you might set up your experiment, although if you have two independent samples, there is some extra work involved if you have different sample sizes or one sample varies more than the other. In those cases you’re probably better off using a slight variation on the t-test called Welsh’s t-test.

In our case, we are comparing the temperature and humidity readings of two different sensors over time, so we can pair our data as long as the sensors are read at more or less the same time. Our null and alternate hypotheses are straightforward here: the sensors either don’t produce significantly different results, or they do.

The two DHT11 sensors were taped down to my desk. They were read with a NodeMCU and the data pushed to a ThingsBoard server.

Next, we can sample. The readings from both sensors were taken at essentially the same time every 10 seconds, and sent via MQTT to a Thingsboard server. After a couple of days, the average temperature recorded by each sensor over 10 minute periods was retrieved. The sensor doesn’t have great resolution (1 °C), so averaging the data out like this made it less granular. The way to do this is sort of neat in ThingsBoard.

First you set up an access token:

$curl -X POST --header 'Content-Type: application/json' --header 'Accept: application/json' -d '{"username":"yourusername", "password":"yourpassword"}' 'http://host.com:port/api/auth/login'

Then you request all data for a particular variable, averaged out every 10 minutes in JSON format (timestamps will be included):

$curl -v -X GET "http://host.com:port/api/plugins/telemetry/DEVICE/devicekey/values/timeseries?keys=variablename&startTs=1510917862000&endTs=1510983920000&interval=600000&limit=10000&agg=AVG" \
--header "Content-Type:application/json" \
--header "X-Authorization:Bearer (token goes here)" > result.txt

What’s cool about using an API like this is that you can easily automate data management and testing as parts of a decision engine. If you’re using less accurate sensors, or are just measuring something that varies a lot, using statistical significance as the basis to make a decision instead of a single sensor value can really improve reliability. But I digress, back to our data!

Next, I did a little data management: the JSON was converted to a CSV format, and the column titles removed (timestamp and temperature). That made it easier for me to process in Python. The t-test assumes normally distributed data just like the z-test does, so I loaded the data from the CSV file into a list and ran the test:

import scipy.stats as stats
import csv
import math as math
import numpy as numpy
#Set up lists
tempsensor1=[]
tempsensor2=[]
#Import data from a file in the same folder
with open('temperature1.csv', 'rb') as csvfile:
datareader = csv.reader(csvfile, delimiter=',', quotechar='|')
for row in datareader:
tempsensor1.append(float(row[1]))
with open('temperature2.csv', 'rb') as csvfile:
datareader = csv.reader(csvfile, delimiter=',', quotechar='|')
for row in datareader:
tempsensor2.append(float(row[1]))
#Subtract one list from the other
difference=[(i -j) for i, j in zip(tempsensor1, tempsensor2)]
#Test for normality and output result
normality = stats.normaltest(difference)
print "Temperature difference normality test"
print normality

In this case the normality test came back p>0.05, so we’ll consider the data normal for the purposes of our t-test. We then run our t-test on the data with the below. Note that the test is labeled ‘ttest_1samp’ in the statistics package – this is because running a 1-sample t-test on the difference between two datasets is equivalent to running a paired t-test on two datasets. We had already subtracted one list of data from the other for the normality test above, and now we’re checking if the result is significantly different from zero.

ttest = stats.ttest_1samp(difference, 0, axis=0)
mean=numpy.mean(difference)
print "Temperature difference t-test"
print ttest
print mean

The test returns a t-test statistic of -8.42, and a p-value of 1.53×10-13, which is much less than our threshold of p=0.05. The average difference was -0.364 °C. What that means is that the two sensors are producing significantly different results, and we have a ballpark figure for what the difference should be at a temperature of around 30 °C. Extrapolating that result to very different temperatures is not valid, since our data only covered a small range (29-32 °C).

I also ran the above test on humidity data, but the results aren’t interesting because according to the datasheet (PDF warning), the relative humidity calculation depends on the temperature, and we already know the two devices are measuring significantly different temperatures. One interesting point was that the data was not normally distributed – so what to do?

A commonly used technique is just to logarithmically transform the data without further consideration and see if that makes it normally distributed. A logarithmic transformation has the effect of bringing outlying values towards the average:

difference=[(math.log1p(i) - math.log1p(j)) for i, j in zip(humidity1, humidity2)]
normality = stats.normaltest(difference)
print "Humidity difference (log-transformed) normality test"
print normality

In our case, this did in fact make the data sufficiently normally distributed to run a test. However, it’s not a very rigorous approach for two reasons. First, it complicates exactly what you are comparing (what is the meaningful result if I compare the logarithm of temperature values?). Secondly, it’s easy to just throw various transformations at data to cover up the fundamental fact that your data is simply not appropriate for the test you’re trying to run. For more details, this paper points out some of the problems that can arise.

A more rigorous approach that is increasing in popularity (just my opinion on both counts), is the use of non-parametric tests. These tests don’t assume a particular data distribution. A non-parametric equivalent to the paired t-test is the Wilcoxon signed-rank test (for unpaired data use the Wilcoxon rank-sum test). It has less statistical power than a paired t-test, and it discards any datum where the difference between pairs is zero, so there can be significant data loss when dealing with very granular data. You also need more samples to run it: twenty is a reasonable minimum. In any case, our data was sufficient, and running the test in Python was simple:

import scipy.stats as stats
list1=[data__list_goes_here]
list2=[data__list_goes_here]
difference=[(i -j) for i, j in zip(list1, list2)]
result=stats.wilcoxon(difference, y=None, zero_method='wilcox', correction=False)
print result

When we ran it, the measured humidity difference was significant, with an average difference of 4.19%.

You might ask what the practical value of all this work is. This may just have been test data, but imagine I had two of these sensors, one outside my house and one inside. To save on air conditioning, a window fan turns on every time the temperature outside is cooler than the temperature inside. If I assumed the two devices were exactly the same, then my system would sometimes measure a temperature difference when there is none. By characterizing the difference between my two sensors, I can reduce the number of times the system makes the wrong decision, in short making my smart devices smarter without using more expensive parts.

As a side note, it has been overstated that it’s easy to lie with statistics. To borrow an idea from Andrejs Dunkels, the real issue is that it’s hard to tell the truth without them.


Filed under: hardware, how-to

Car Lights for Reflow Heat Source

via hardware – Hackaday

If you only have a car and you need to unsolder some tricky surface mount components: what would you do? If you’re Kasyan TV, you’d remove your car’s halogen lights and get to town. That’s right: car lights for reflow.

When the friend of the host of Kasyan TV needed to remove some roasted toasted FETs from his motherboard but didn’t have anything for reflowing, she took some headlights and used them as an infrared source to desolder the FETs. Powered by a lab supply (although car batteries work too), the process works with 60 and 100-watt bulbs.

Now, reflowing with halogen bulbs isn’t new, and we’ve seen it done with the run of the mill 100-watt bulbs and a halogen floodlight. However, what we really like about using car lights is that they’re available everywhere and we already own some that we could (temporarily) repurpose. Now, don’t get us wrong – if you’re going to be reflowing more than just a little, there are plenty of alternative methods that don’t involve staring at “rather bright lights” for extended periods of time.

People ’round these parts can’t seem to get enough of reflow: from open source reflow oven controllers to reflowing with a hair straightener we’ve seen quite a bit. If you’re new to the reflow arena, we’ve got zero to hero: reflow style just for you. And if DIY at home reflow isn’t intense enough for you, we’ve got next level reflowing as well.

The full video is after the break, complete with Kasyan TV’s sponsored segment in the middle..


Filed under: hardware, how-to

Laser Cutter Alignment Mod Skips Beam Combiner

via hardware – Hackaday

A lot of the DIY laser engravers and cutters we cover here on Hackaday are made with laser diodes salvaged from Blu-ray drives and projectors, which are visible lasers in the 400 – 450nm range (appearing as violet or blue). Unfortunately there is an upper limit in terms of power on visible diode lasers, most builds max out at 5W or so. If you need more power than that, you’ll likely find yourself looking at gas laser cutters like the K40. While the K40 is a great starting point if you’re looking to get into “real” lasers, it’s a very different beast from the homebrew builds using visible lasers.

With a gas laser the beam itself is invisible, making it much more difficult to align or do test runs. One solution is to add a visible laser to the K40 which can be used to verify alignment, but making sure it’s traveling down the same path as the primary laser usually requires an expensive beam combiner. Looking to avoid this cost, [gafu] wanted to see if it was possible to simply move the visible laser into the path of the primary beam mechanically.

An adjustable microswitch detects when the lid has been opened.

In the setup that [gafu] has come up with, a cheap laser module (the type from a handheld laser pointer) is moved into the path of the primary laser on an arm that’s actuated by a simple hobby servo. To prevent the primary and visible lasers from firing at the same time, an Arduino is used to control the servo given the current state of the K40’s lid. If the lid of the K40 is open, the primary laser is shutoff and the visible laser is rotated into position so the operator can see where the primary laser’s beam would be hitting. Once the lid is closed, the visible laser rotates out of the way and the primary is powered back up.

Running the cutting or engraving job with the lid of the K40 machine open now let’s [gafu] watch a “dry run” of the entire operation with the visible laser before finally committing to blasting the target with the full power beam.

We’ve covered many hacks and modifications for everyone’s favorite entry-level CO2 laser cutter. From replacing the controller to making it bigger, K40 owners certainly seem like a creative bunch.


Filed under: Arduino Hacks, hardware, Laser Hacks

Mike Harrison at the Superconference: Flying LCD Pixels

via hardware – Hackaday

Mike Harrison, perhaps better known to us as the titular Mike of YouTube channel mikeselectricstuff, is a hardware hacking genius. He’s the man behind this year’s Superconference badge, and his hacks and teardowns have graced our pages many times. The best thing about Mike is that his day job is designing implausibly cool one-off hardware for large-scale art installations. His customers are largely artists, which means that they just don’t care about the tech as long as it works. So when he gets together with a bunch of like-minded hacker types, he’s got a lot of pent-up technical details that he just has to get out. Our gain.

He’s been doing a number of LCD installations lately. And he’s not using the standard LCD calculator displays that we all know and love, although the tech is exactly the same, but is instead using roughly 4″ square single pixels. His Superconference talk dives deep into the behind-the-scenes cleverness that made possible a work of art that required hundreds of these, suspended by thin wires in mid-air, working together to simulate a flock of birds. You really want to watch this talk.

Did you watch the talk? My favorite mechanical trick that Mike shared was the way he held the LCDs between two PCBs — milling a winding path through them with a 1.6 mm milling to tightly clamp down on the 1 mm thick LCD glass. Getting a black PCB is easy enough, but how do you get black FR4? Well, you could custom order it at great expense, or you could use (wait for it) a black Sharpie marker.

If you’ve never played around with LCDs before, scroll back up and watch the talk, because he also presented tons of details about the electrical requirements of doing so. LCD panels need AC drive, which for ICs that run on a single-sided supply means reversing the drive direction somehow. Mike was able to use a simple microcontroller with a PWM and inverted PWM signal to generate the “AC” and allow for beautiful greyscale on the LCD pixels. Bet your calculator doesn’t do that. (But it could!)

A lot of the details of the design were driven by the fact that these pixels needed to “float” in mid-air. To achieve this effect, they were suspended on two stainless steel wires, and the data and power were sent down the same lines. With up to five meter long wires, Mike was afraid of the data being sent down them radiating out from these unintentional antennas, as well as interference coming in. A 20 kHz Manchester encoding delivered the data. This is a win because it guarantees the maximum possible length of time between positive pulses, which is important because these pulses keep the devices powered. The resulting circuit is so well thought-out that every part is crucial to its success, and is a beauty to behold.

Installation of a large-scale piece like this could be a logistical nightmare. This is not Mike’s first rodeo. He designed in features like power and data OK LEDs that are visible from the vantage point of an installer, making it possible to individually troubleshoot each row of LCDs at a glance.

One of the deepest tips is how Mike maps the device ID of each panel into the controller that needs to know about their physical location. The hard way to do this is program device number one, hang it, program device number two, hang it, and repeat. It’s a bookkeeping nightmare. Instead, the installer presses a “button” on a single device, the controller sends the address down the bus, and the device accepts that address as its own. This means that the addresses can be assigned after everything is set up, which also makes replacement a piece of cake. But the real beauty of this system is the “button” — a tactile switch, magnetic sensor, light sensor — you can use whatever input you’ve got on the node. Mike built an installation that already had accelerometers built in, and tapped each node to lock in the IDs.

Here, none of these were a good match for the minimal hardware on each panel and the high-in-the-air nature of the setup. Mike wanted a system that could be deployed on the end of a pole, but that wouldn’t require electrical contact, which meant something like capacitive sensing. Rather than rely on the tiny capacitive changes of a finger, he mounted a CCFL inverter on a stick, and the LCD panels detected the resulting field. The final touch was that the controller could sense the current each node used, so the LCD pixel could signal that it had received the new ID and the whole procedure could proceed at the speed that someone could wave the wand over the panels. Very slick.

It’s always a pleasure to hear Mike speak on what he’s truly passionate about — clever solutions to hardware problems. I learn something new, and useful, every time.


Filed under: cons, hardware

MIT Is Building a Better 3D Printer

via hardware – Hackaday

Traditional desktop 3D printing technology has effectively hit a wall. The line between a $200 and a $1000 printer is blurrier now than ever before, and there’s a fairly prevalent argument in the community that you’d be better off upgrading two cheap printers and pocketing the change than buying a single high-end printer if the final results are going to be so similar.

The reason for this is simple: physics. Current printers have essentially hit the limits of how fast the gantry can move, how fast plastic filament can pushed through the extruder, and how fast that plastic can be melted. To move forward, we’re going to need to come up with something altogether different. Recently a team from MIT has taken the first steps down that path by unveiling a fundamental rethinking of 3D printing that specifically addresses the issues currently holding all our machines back, with a claimed 10-fold increase in performance over traditional printing methods.

MIT’s revolutionary laser-assisted hot end.

As anyone who’s pushed their 3D printer a bit too hard can tell you, the first thing that usually happens is the extruder begins to slip and grind the filament down. As the filament is ground down it starts depositing plastic on the hobbed gear, further reducing grip in the extruder and ultimately leading to under-extrusion or a complete print failure. To address this issue, MIT’s printer completely does away with the “pinch wheel” extruder design and replaces it with a screw mechanism that pulls special threaded filament down into the hot end. The vastly increased surface area between the filament and the extruder allows for much higher extrusion pressure.

An improved extruder doesn’t do any good if you can’t melt the incoming plastic fast enough to keep up with it, and to that end MIT has pulled out the really big guns. Between the extruder and traditional heater block, the filament passes through a gold-lined optical cavity where it is blasted with a pulse modulated 50 W laser. By closely matching the laser wavelength to the optical properties of the plastic, the beam is able to penetrate the filament and evenly bring it up to nearly the melting point. All without physically touching the filament and incurring frictional losses.

There are still technical challenges to face, but this research may well represent the shape of things to come for high-end printers. In other words, don’t expect a drop-in laser hot end replacement for your $200 printer anytime soon; the line is about to get blurry again.

Speeding up 3D printing is a popular topic lately, and for good reason. While 3D printing is still a long way off from challenging traditional manufacturing in most cases, it’s an outstanding tool for use during development and prototyping. The faster you can print, the faster you can iterate your design.

Thanks to [Maave] for the tip.


Filed under: 3d Printer hacks, hardware, Laser Hacks

Finished Dryer Will Text You

via hardware – Hackaday

Here’s a slightly different way to check on the status of your laundry. Instead of checking if the machine is vibrating, or listening for sound, or pulling everything apart and hacking an ESP8266 into it, check the power that the machine is drawing. This is what [Scrand] did in his IoT dryer build.

The secret behind the hack is the Sonoff POW, a small device that sits in between the wall and the dryer. It has a relay in it that controls it, but, importantly for this hack, it’s able to measure the power consumption used by what’s plugged into it. By installing the ESPurna firmware on it, he can now use all the power of the firmware to control and monitor what’s connected to the POW. He wrote a PowerShell script to monitor the http server now running on the POW checking on how much power is being drawn by the dryer. When that power drops, the laundry is done, and in the case of [Scrand], a text is sent saying so.

When you’re sitting on the couch relaxing, why get up every five minutes to check your laundry when you can have it text you when you know it’s done? Then you can decide whether to get up and deal with it or just leave it until later. The whole reason ESPurna exists to begin with is to check on the status of the laundry.  Or, you can go a bit overboard with this laundry room monitor.


Filed under: hardware, home hacks