How to connect Flir Lepton to NVIDIA Jetson TK1

person kasunfolder_openUncategorizedaccess_time November 20, 2016

Introduction

In this post, I will discuss how to connect FLIR Lepton module with NVIDIA Jetson TK1.If you are new to Jetson TK1 and struggling on how to kick off the development with that , you can refer to my previous post to get to know about how to get started with NVIDIA Jetson TK1.In this post I assume, you have already installed Operating system(Ubuntu 14.04) using Jetpack or any other way.

There is an excellent guide on how to enable SPI with Jetson TK1 and connect to Arduino Due, It helped me a lot understanding  the hardware capabilities of Jetson TK1 to enable SPI communication , and how kernel device drivers and device tree helps to allow communication with SPI hardware in TK1 with programmers in Operating system level.It is better to first to go through that article up to part 3.My Jetson TK1 setup is Ubuntu 14.04 installed with Jetpack R21.5 and update the kernel with Grinch 21.3.4 for Jetson TK1.If you have already installed jetpack and running the OS following topic will cover how to install Grinch 21.3.4 kernel, and configure to use SPI.

Update Kernel

I actually wanted to connect Lepton with Jetson without doing any Kernal modification , so I tried building official kernel enabling the SPI module but It didn’t work and ended up with hanging the OS after few seconds of booting. So re-flash the Jetson and tried with latest Grinch kernel.As the steps mention here updating the stock kernel with Grinch kernel is easy.If you have fresh Jetpack installation(In my case Jetpack 21.5). Then run the following commands to update the kernel.

  1. Download required firmware,kernel modules and boot zimage files

2. Make sure you check the MD5 hash with following (It sometimes download 135K junk file, nothing in it)

3. Then extract and copy the files to system

4. If you are following original NVIDIA dev post sofa , stop here and the coping DTB file into /boot directory, Because we have to make some changes before copying the device tree blob file into /boot

  • The reason is , we need to change the frequency of the SPI clock or reduce the default Jetson touch SPI CLK frequency to comfortable range to work with Lepton, For that we will decompile the .dtb file and generate the .dts(Device tree source file) and do the modification to the .dts file and the compile the source code using dtc(Device Tree Compiler) and copy the modified device tree blob(.dtb) file to /boot directory

So for doing the above task first we need the dtc tool, The source code for the dtc comes with the kernel it self in the kernel/scripts/dtc/ you can compile it and use.Else I have uploaded here the pre-compiled dtc tool for Jetson TK1 , you can use that if you don’t want to do any kernel builds.

  1. One you have the dtc tool , go to the directory where you have downloaded the tegra124-jetson_tk1-pm375-000-c00-00.dtb file and run the following command
  2. Then open the *.dts file from your favourite text editor and locate the section [email protected] and change the below highlighted lines,

    I have only change the original 0x17D7840 value to 0xF42400, setting the max frequency value from 25MHz to 16MHz
  3. Then compile the device tree source file again and copy it to /boot directory, to do that use the following command

As for the instruction in the Nvidia forum , You should get the /dev/spidev0.0 after a restart , But it seems it’s not working after Jetpack ~R21.3 onwards.So I google for a solution for this and finally found a workaround to get the SPI work

Fixing conflicts with Jetson TK1 SPI touch drivers

After following exact steps as mentioned in neurorobotictech article and grinch kernel update post , I still couldn’t get the /dev/spidev0.0 appear in the device directory, So to check what is going on with the kernel , I check for spi logs in dmesg and tried to find a clue  dmesg | grep "spi"  it shows that

But it was quite confident that , I haven’t using that port for any reason,So I further look for what could use this port other than spidev, and finally was able to find the answer from this post

This is because the touch controller fails to start up, becuase MCP251x has taken it already. This can be ignored, or you can disable the
touch controller by editing /boot/extlinux/extlinux.conf and setting “[email protected]

So simply open the  /boot/extlinux/extlinux.conf file and change the touch_id value from [email protected] to [email protected], And that’s it.Now restart the jetson and check whether the /dev/spidev0.0 is there , and most probably you should be able to locate it now.

So let’s discuss how to connect the FLIR wire connections physically into Jetson TK1 GPIO adapter.

Connecting Hardware

We have bought an FLIR Lepton module with the pureengineering lepton breakout board attach from the groupgets

IMG_20161120_112012

When I tried to connect the Lepton breakout board to Jetson , the harder part was to find the correct pins for SPI0 in Jetson J3A1 pad, Luckily descriptive explanation about all the GPIO port in Jetson was here. Then it was just a matter of picking up the correct pin from Jetson and connect to Lepton via a jumper cable, To make the process easy I have marked the correct pin number and it’s named from above elinux wiki and the pin name appear in breakout board along with the pin number from right to left starting from SCL. 

Before start connecting the lines , Make sure you have power off the jetson and unplug it from the power supply.

Jetson_lepton_pin_config_low

Now connect the cables and start the jetson TK1 to proceed with the next step, Capturing the data.

BTW Is I2C connections missing ? , No you can simply connect the SCL and SDA lines from Lepton breakout board to 18 GEN2_I2C_SCL_3.3V  and 20 GEN2_I2C_SDA_3.3V and get the I2C working, But I haven’t tested this yet.

Software setup for capturing data from Lepton

After connecting the device , I still didn’t know that whether I have connected the device to right pins or even all the above steps have enabled the SPI in jetson, The next challenge was to make the connection between hardware and software.I tried with the pylepton library from groupgets GitHub repo, and need to do some slight changes to get it work.

One thing is we need to change the SPEED constant value in Lepton.py to  SPEED = 16000000 to make it comply with the max_frequency value we specified in device tree source, And also we need to change the MODE constant value to MODE = SPI_MODE_0  Because according to Jetson TK1 technical reference manual(TRM) ,SPI0 in jetson works in SPI mode0. After making those changes run the pylepton_capture script with sudo permission and you wold be able to take a thermal grayscale image from Lepton.

image_sample_33 image_sample_44 image_sample_74 image_sample_127 image_sample_144

Here you can find some raw sample jpg images along with the value received from the Lepton before converting the values to 0-255 range.

What next. . .

Next, I will write a post on how to use the thermal data(Not specifically image just raw data) and train a neural network model using Caffe deep learning library to detect human presence.Well, there is a lot of work there in the previous phrase.Will try to put them under one post.Since then good luck connecting the lepton to Jetson 🙂

References :

 

  • https://goo.gl/6sqUBH
  • http://www.electricstuff.co.uk/lepton.html
  • https://github.com/groupgets/pylepton
  • https://devtalk.nvidia.com/default/topic/740390/?comment=4421685
  • http://elinux.org/Jetson/Installing_OpenCV
  • http://neurorobotictech.com/Community/Blog/tabid/184/ID/13/Using-the-Jetson-TK1-SPI–Part-3-Configuring-SPI-in-the-device-tree.aspx
  • https://github.com/chneukirchen/linux-jetson-tk1/blob/master/Documentation/devicetree/bindings/spi/spi-bus.txt
  • https://learn.sparkfun.com/tutorials/serial-peripheral-interface-spi

Comments

  1. I followed these steps and can not seem to get the spidev0.0 to come up in the dev folder. I only get spidev0.1, can anyone help? I havent found a solution to this. I am running Tk1 21.5 and followed this from top down

    1. for those looking for a solution to this problem going through the source code and editing the pylepton software to look for /dev/spidev0.1 by default rather than /dev/spidev0.0 will work.

      1. Thanks for the information.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code class="" title="" data-url=""> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong> <pre class="" title="" data-url=""> <span class="" title="" data-url="">