Skip to main content

Iterating faster

Today I spent a few minutes writing a Makefile. I think they may turn out to be some of the most effective minutes I've ever spent.

At work we build using SCons. When I change a single source file, rebuilding takes between 50 and 65 seconds. That's more than long enough to lose my focus and get distracted, so to stay productive I have to work in larger batches. I code in longer bursts and recompile less often, so that the ratio of productive time to down time stays up; unfortunately that also means fewer opportunities to test my changes.

That's not ideal, but it's manageable much of the time. Not at the moment though. Right now I'm developing an algorithm where I need to be able to quickly try out different approaches for each of the steps. In this situation I really want to see the effect of each change in isolation, so I can't batch up my coding. All of a sudden those 50 second build times are a real problem.

I spent a little while looking into ways to speed SCons up and managed to get the rebuild time down to around 45 seconds. Still nowhere near fast enough.

Fortunately I'm working in a very self-contained part of our code-base, so I could limit the amount of files I had to consider. Re-running SCons with verbose output turned on gave me the exact command being used to compile each of them, so I saved those to a shell script. Rebuild time with this shell script was down to about 15 seconds: a huge improvement, but still not quite quick enough. This script was recompiling every file each time I ran it though - surely I could do something a bit smarter than that...?

Enter make. It knows how to check for changed files & I've used it enough that I can put together a Makefile pretty quickly. Factoring out the commands from the shell script into rules and adding enough dependency information to be useful, I had it set up in a couple of minutes. The result: most rebuilds are now around 3 seconds. Perfect.

So now, not only am I saving almost a whole minute of wasted time with each rebuild, I've all but removed the chance to get distracted and also opened the door on a more effective way of working. I can make small changes and rebuild to see the effect of each in isolation. I can develop my algorithm a lot faster now, because I can iterate faster; and that's why I think they'll be some of the most effective minutes I've ever spent.

Comments

Post a Comment

Popular posts from this blog

Triangle bounding boxes in a single byte

Just thought of a way to store the bounding box for a single triangle in only one byte. It's not really practical or something you'd ever really want to use, but what the hell. Assume we have some kind of indexed mesh structure with a list of vertex positions and a list of triangle indices:   struct Mesh {     std::vector<vec3> verts;     std::vector<uvec3> triangles;   }; We can find the bounding box of a triangle by taking the min and max of all three vertices:   vec3 Mesh::lowerBound(uint32_t tri) const {     vec3 v0 = verts[triangles[tri].x];     vec3 v1 = verts[triangles[tri].y];     vec3 v2 = verts[triangles[tri].z];     return min(min(v0, v1), v2);   }   vec3 Mesh::upperBound(uint32_t tri) const {     vec3 v0 = verts[triangles[tri].x];     vec3 v1 = verts[triangles[tri].y];     vec3 v2 = verts[triangles[tri].z];     return ...
2 comments
Read more

LD_DEBUG

Posting this mainly as a reminder to myself... If you ever find yourself needing to figure out a dynamic library loading problem on Linux, LD_DEBUG can be a massive help. This is an environment variable you can set to make the dynamic linker print out a ton of useful diagnostic info. There are a number of different values which control the amount and type of diagnostics printed. One of the values is help; if you set LD_DEBUG to this and run executable it will print out a list of all the available options along with brief descriptions. For example, on my Linux workstation at the office: > LD_DEBUG=help cat Valid options for the LD_DEBUG environment variable are: libs display library search paths reloc display relocation processing files display progress for input file symbols display symbol table processing bindings display information about symbol binding versions display version dependencies all all previous options combi...
Post a Comment
Read more

Assert no lock required

This is a technique I learnt about from Jason Gregory's excellent book, Game Engine Architecture (3rd Edition) . If you have a shared resource accessed by multiple threads, where you're fairly certain that it's only ever accessed by one thread at a time, you can use an assert() to check for this at debug time without having to pay the runtime cost of locking a mutex. The implementation is fairly straightforward: class UnnecessaryMutex { public: void lock() { assert(!_locked); _locked = true; } void unlock() { assert(_locked); _locked = false; } private: volatile bool _locked = false; }; #ifdef ENABLE_LOCK_ASSERTS #define BEGIN_ASSERT_LOCK_NOT_REQUIRED(mutex) (mutex).lock() #define END_ASSERT_LOCK_NOT_REQUIRED(mutex) (mutex).unlock() #else #define BEGIN_ASSERT_LOCK_NOT_REQUIRED(mutex) #define END_ASSERT_LOCK_NOT_REQUIRED(mutex) #endif Usage is equally straightforward: UnnecessaryMutex gMutex; void PossiblyOverlappingFunction...
Post a Comment
Read more