Automating C++ builds in ROS

By Thomas Weng on September 9, 2017
~3 min. read

If you’re working on a C++ ROS project, you probably run catkin build every time you make a change. This is tedious and takes you out of your programming flow. It’s especially annoying when your build fails multiple times due to small errors. I’m a big proponent of keeping the iteration loop as small as possible.1

To fix this, I’ve automated the build process to build when saving a file! No more manual building :).

Here’s how it works. I’ve written a shell script called that kicks off a build for you every time you save a file in your source directories. If the build fails, it will output the error. If the build succeeds, it’ll print a success message. Here’s an example of a build failure, followed by success:

$ bash
Setting up watches.
Watches established.
octomapper.cpp modified, rebuilding...
Errors     << perception:make /home/tweng/catkin_ws/logs/perception/build.make.1447.log
/home/tweng/catkin_ws/src/ros-project/src/perception/src/octomapper.cpp: In member function ‘void perception::Octomapper::publish_octomap(octomap::OcTree*)’:
/home/tweng/catkin_ws/src/ros-project/src/perception/src/octomapper.cpp:99:3: error: ‘pub’ was not declared in this scope
make[2]: *** [CMakeFiles/perception_octomapper.dir/src/octomapper.cpp.o] Error 1
make[1]: *** [CMakeFiles/perception_octomapper.dir/all] Error 2
make: *** [all] Error 2
cd /home/tweng/catkin_ws/build/perception; catkin build --get-env perception | catkin env -si  /usr/bin/make --jobserver-fds=6,7 -j; cd -
Failed     << perception:make           [ Exited with code 2 ]

There is syntax highlighting in your terminal which makes this output more readable.

After fixing the issue and saving, the build runs again automatically:

octomapper.cpp modified, rebuilding...
[build] Summary: All 2 packages succeeded!

It’s fairly simple to get this set up for your workspace. You’ll need to:

  1. Get the script from this Github gist:
  2. Configure it to watch your workspace directories
  3. Run it in a terminal using bash
  4. Start coding and enjoying ~build-on-save~

But wait, there’s more: roslaunch auto-restart!

After your build completes, you’ll probably need to run or restart your ROS nodes to test your changes. That’s another manual step we can automate.

This time, a script called runs your project’s roslaunch command and listens periodically to make sure your ROS nodes are alive. As you make changes and get a successful build, – the original script – sends a signal to kill your ROS nodes.2 When the ROS nodes die, will automatically restart them, grabbing your newest build. Here’s an example of what restarting looks like:

$ bash
Launching roslaunch
... logging to /home/tweng/.ros/log/3674ab20-73be-11e7-b57f-b8ca3ab4b589/roslaunch-silverarm-15112.log
Checking log directory for disk usage. This may take awhile.
Press Ctrl-C to interrupt
Done checking log file disk usage. Usage is <1GB.

started roslaunch server http://localhost:39545/


/process_cloud_main shutdownCallback:163: Shutdown request received.
/process_cloud_main shutdownCallback:164: Reason given for shutdown: [user request]
================================================================================REQUIRED process [process_cloud_main-1] has died!
process has finished cleanly
log file: /home/tweng/.ros/log/3674ab20-73be-11e7-b57f-b8ca3ab4b589/process_cloud_main-1*.log
Initiating shutdown!
[publish_saved_cloud-3] killing on exit
[person_broadcaster-2] killing on exit
[process_cloud_main-1] killing on exit
shutting down processing monitor...
... shutting down processing monitor complete
done notices that the node has gone down and triggers a restart:

Launching roslaunch
... logging to /home/tweng/.ros/log/3674ab20-73be-11e7-b57f-b8ca3ab4b589/roslaunch-silverarm-30141.log
Checking log directory for disk usage. This may take awhile.
Press Ctrl-C to interrupt
Done checking log file disk usage. Usage is <1GB.

started roslaunch server http://localhost:34414/

You can get here. You’d run it in a terminal (bash, just like the first one.

If your builds take a lot of processing power and/or take a long time, you may need to make adjustments to this script. I personally haven’t had any problems rebuilding on every save. One option if you do have problems is to rebuild only the package you are working on, and not the whole workspace.

Hope this helps and you find it useful!


  1. Originally inspired by Brett Victor’s talk, “Inventing on Principle.” Check it out a recording of it here: video link

  2. I set the main node with the required=true attribute in my launch file so I only need to kill that node to stop the others. 

How I Journal

By Thomas Weng on May 30, 2017
~3 min. read

Can you remember, off the top of your head, what you were doing at this date and time two months ago? What about just two weekends ago? It is surprising to me how rarely people journal when our memories are so ephemeral. We have new thoughts, experiences, and conversations every day, yet we save and reflect on so few of them.

Journaling is one effective way to take control of your life and memories. Although taking photos and videos are more common ways to keep track of what you do or see, I’ve found writing to be more effective when it comes to reflecting on my life and staying organized.

Forming habits

People I’ve spoken to on this subject are generally receptive to the concept of journaling, but many find it difficult to prioritize the activity and keep a routine. I personally journal about once a week, filling in several days at a time. After a few months, it’s fulfilling to go back and revisit old entries. It’s amazing how a few short sentences can bring you back to an amazing conversation with a close friend, or your mood on a sunny, relaxing day, for example.

For me, one of the biggest starting hurdles was choosing the proper journaling medium. There are countless apps out there, but many of them have inconvenient tradeoffs. Apps often make it difficult to export your data, get in your way with ads, or keep useful features behind paywalls. There’s always pen and paper, but while writing by hand can be relaxing, it is also slower than typing, difficult to search through, and hard to preserve through the years.

Journaling in Markdown

My solution for the past year and a half has been to journal digitally in Markdown files. Markdown is designed to be easy-to-read, easy-to-write, and it avoids the pitfalls I mentioned above. Here’s a example of how I journal, albeit somewhat contrived and with less detail:

## Mon 29
* __exercise__ 7 minute workout
* Skyped with X. It was the first time we caught up in a while! We talked about our living situations, our jobs, and what we've been spending our spare time doing these past few months.
* Met up with Y at Cafe Cesura. Worked on a blog post about journaling.

Which converts into the following HTML:

Mon 29

  • exercise 7 minute workout
  • Skyped with X. It was the first time we caught up in a while! We talked about our living situations, our jobs, and what we’ve been spending our spare time doing these past few months.
  • Met up with Y. at Cafe Cesura. Worked on a blog post about journaling.

Markdown’s lightweight structure provides both organization and flexibility. I generally use headers for dates, followed by bullet points describing each significant event of the day. I keep each month in a separate file, and each year of files in a distinct folder.

I also use tags like __exercise__ above to mark certain categories of events or thoughts. This has an added benefit of being easily searchable in Atom, my text editor of choice for journaling.

Atom also provides a handy Markdown preview:

Using Markdown, I can write faster than on paper, search through my notes quickly, and save backup copies easily. I don’t have to rely on any apps, nor give any third parties ownership of my data.

A year and a half of Markdown

As I mentioned earlier, I’ve been journaling in markdown for the past 1.5 years, and I’m really enjoying the setup. I journal approximately once a week, but I’ve been trying to increase the frequency. The amount of information I retain about a particular day significantly diminishes by the third passing day.

Perspectives on time and value

Journaling has given me a much better higher-level understanding of where my time goes and more importantly, what I value. Keeping the one folder per year structure I have now, I’ll have 60 folders of journal files, give or take, by the end of my life. That’s not a lot, all things considered, and made me think about what I want to accomplish and record in my lifetime.

Handwritten notes are still cool

Digital journaling hasn’t completely replaced pen and paper. I still take handwritten notes in pocket notepads to remember things to do, books to read, or random thoughts. More often than not, I’ll transfer the interesting portions into my journal.

What’s next

There are other sources of data besides journaling that I want to pull together to form a richer picture. Photos, calendars, conversations, and geotags would add a lot of context to my journal.

To this end, I started writing a parser in Python for the Markdown-generated HTML files. The output is a Python object, which I can work with programmatically. As an example use case, I’ll be able to match up a given date with photos taken on that date in Google Photos. Ultimately, I hope to put together a simple webpage to show what I was up to on a given date in every year of my life.

If you liked this post, you may also like this one I wrote on how I keep a calendar.

Getting Started with Tensorflow

By Thomas Weng on February 10, 2017
~1 min. read

I took some time out this past weekend to work through an introductory machine learning talk on building models using Tensorflow. Within a few hours, I coded up a model classifying handwritten digits from the MNIST dataset with 99.5% accuracy! Considering that the accuracy of cutting edge research models currently hover at around 99.7%, my model was a pretty good result for just a few hours of development.

The resource I relied on to ramp up was Google’s crash course through Tensorflow, presented by Martin Gorner at Devoxx. The talk starts with a simple one-layer neural network, building quickly into multi-layer neural nets, convolutional neural nets, recurrent neural nets, and a collection of optimization techniques (e.g. learning rate decay, dropout, batch normalization).

I took the time to write out the code and get things working on my machine. That took longer than simply watching the video, but it deepened my understanding and guaranteed that I didn’t gloss over any important details. The slides from the talk included most of the code required for the models, but I did need to fill in some of the gaps. When I got stuck, I referenced the complete model at the Github repo Gorner created for this talk.

Gorner’s presentation glosses over the math behind concepts presented in order to focus on model building. I think this was a smart tradeoff. Having spent quite a bit of time taking courses and learning the math powering machine learning, the details of how the math works would probably be too involved for a short talk. And when it comes to actually building models, Tensorflow does the heavy lifting and abstracts the mathematical details away anyway.

Of course, the math does become more relevant as one continues working with ML. I’m sure I wouldn’t have picked things up so quickly with Tensorflow had I not already studied the background concepts.

I’m looking forward to playing more with Tensorflow and learning about what’s going on under the hood. I’ll be keeping up with the math through online resources – courses, textbooks, and hopefully papers. Most of all, I’m excited to take the training wheels off soon and start building my own models from scratch!

Robotics Day at Microsoft

By Thomas Weng on February 1, 2017
~3 min. read

When it comes to applying for opportunities, I’m no different from everyone else: I always get anxious. But preparing an application is often a great way to make your work presentable and gauge your progress.

Earlier this month, I heard about Robotics Day, an all-day event at Microsoft consisting of a series of talks, guided tours of Microsoft Research labs, and an expo showcasing robots built by hobbyist employees. The organizers sent out a call for volunteers and an application for prospective expo presenters.

This event was right up my alley. I was excited at the prospect of connecting with other MS folks doing robotics. I had even been building my own robot these past few months, so the expo was the perfect opportunity to show it off! However, I didn’t feel confident about applying for the expo, because I feared my work wouldn’t meet the same bar as other presenters. I didn’t want to stick out and look like I didn’t know what I was doing.

I wavered for a bit about applying, but ultimately I figured that I would give it my best shot and let the organizers decide if I made the cut. I also signed up to be a volunteer, so that in case of rejection, I could at least help out and get to know the organizers.

In the end, I got to do it all: I volunteered, attended the talks, and presented at the expo! I’ll focus on my experience at the expo here, so that this post doesn’t get too long.

At the expo

My presentation naturally focused on Rover, the robot that I have been building and programming for almost half a year. Here is a picture of the bot:


I am building Rover to be an intelligent, mobile agent, kind of like a pet. A big part of the fun is seeing how much intelligence I can build using off-the-shelf hardware and open-source software. I’ll be writing up a more detailed post on building and designing Rover soon.

Rover is still a work in progress, so I put together several materials to aid me with the presentation. Here’s a picture of my booth setup:


Here’s an online version of the poster on the wall; click for a full page view:

Presentation materials

  • Poster. On the poster, I mapped out the project goals, system design, roadmap, and key challenges, and other topics. It served as a handy reference when people asked questions, as I could point out the relevant details quickly.
  • Video. I played a short video on loop with clips demoing Rover at different stages of development. I also included demos of robots on the market aiming to be intelligent agents for comparison.
  • Contact info. I laid out some paper strips with project links and my contact information for people to take, in case they wanted to connect with me later.

How presenting went

The expo lasted for two hours, and lots of people stopped by. I ended up having virtually no downtime, which was great! It was rewarding to see people checking out my robot, looking at my presentation materials, and asking me questions about what I had built.

It was interesting to observe what drew people’s attention. My robot was powered on but stationary, and many people were drawn to the booth by the spinning LIDAR on the robot. It turned out that few people knew that fairly inexpensive LIDARs were available for hobby robotics. Once people were engaged, I pitched my vision for Rover as an intelligent pet-like robot, and described the progress I was making towards that goal.

There were a lot of other cool robots on display, like battle robots, mobile robots made with bicycle wheels, and even a foosball-playing robot! All the presenters were really friendly, and we all had a great time demoing our creations.

What I learned

My biggest takeaway from this experience is realizing how important it is to practice pitching my ideas to other people. Though I had been working on my robot for months, I had very little to show for it besides the physical robot.

Preparing to show my project to other people forced me to distill the nebulous thoughts I had been working with into well-formed concepts. Showing my work also allowed me to gauge how people responded to Rover and get their feedback. And practicing my pitch helped me keep my communication skills sharp.

Finally, I’m learning to become less anxious about applying to things. I want to treat applications more as learning opportunities rather than tests. If I feel overly anxious, not only is it counter productive, but it probably means that I’m overestimating how much I’ll lose if I get rejected. It’s better to just keep trying, learning, and growing.

Thanks for reading, and best wishes to you on whatever projects or opportunities you are working towards! Stay tuned for more details about Rover.

Tags: robotics

You Should Use a Password Manager

By Thomas Weng on January 23, 2017
~2 min. read

It’s easy to get lazy and use the same password for multiple account logins. But doing so leaves you much less secure than having different passwords.

Still, it’s near impossible nowadays to remember an ever-expanding number of passwords for different accounts. Last year, I found that I was spending a silly amount of time coming up with passwords, remembering them, typing them in incorrectly, and resetting them.

I consequently started hunting for a solution, and ended up trying out a password manager. It solved my password issues so well, I haven’t looked back!

There isn’t much to a password manager, and using one has made logging into accounts so much easier and safer. Here’s the gist of it in less than two minutes.

How do password managers work?

A password manager works like a safe. Your passwords are securely stored in an encrypted database, and you access them by entering a master password to decrypt the database.

It’s much easier to remember one long, secure password than numerous different ones. And if the safe (encrypted database) is stolen, thieves wont be able read your passwords without the key (master password).

When you need to log in to an account, you simply unlock your database, copy your credentials, and paste them into the login fields.1 No more straining to type complicated passwords.

When you need a new password, password managers create new ones for you that are hard to crack. Many of them also remind you to change your passwords if it’s been a while.

Which password manager should I use?

I use KeePass, a free program. KeePass is open-source, meaning that a community of security experts (and anyone else who is interested) is vetting the source code to verify that it really is secure.

KeePass, however, is not the most user-friendly service out there. I’ve played with LastPass as an alternative. LastPass keeps your passwords in a “cloud” so you can access them anywhere.2 LastPass runs on an ad-based model, making it also “free” to use.

There are also other password management programs and services out there, though I haven’t tried any others personally.

What if I screw up with a password manager and lose all of my passwords?

If you’re worried about losing your passwords and/or aren’t sure if a password manager is for you, you always have the option of easing into it. Start by putting one password into the manager for something that isn’t critically important, so that you get the hang of using it. As you get more comfortable, you can add more and more passwords.

If anything goes wrong, you can still reset your passwords to online accounts the regular way.

I hope you take a few minutes out of your day to give a password manager a try. In retrospect, I can’t believe it took me so long to take the jump–using a manager has been so much easier than the old system of remembering passwords that weren’t as safe anyway.

Edited January 28, 2017

  • Added pictures.
  • Edited “How do password managers work?” section.


  1. Some services even offer “Auto-Type”, which fills in the login fields for you with one click or shortcut combo! 

  2. I get around this problem with KeePass by keeping my password database in a cloud service myself (e.g. Dropbox, OneDrive, etc.).