I am a firm believer in incremental development, with short cycles of prototyping, frequent user demos, and many more or less informal partial tests.
Refer to Sydney Camm : Hurricane, Swift, Hunter, Harrier. And Simodont.
It may come as a surprise to you that I am also a firm believer in writing up a small requirements document before each test, stating at the very least its purpose, and thoroughly documenting the outcome, especially if it was negative. The maxim is : "an undocumented test equals a test never done". And in fact, if a test was not worth writing up, it should never have been done in the first place.
At least maintain this fiction after the fact. Like science in a textbook.
Keep this logical structure alive if you can, even if chronologically and historically, it did not go that way.
But do add footnotes on how it really went.
Methodically document everything in sight, including the history of tests and ideas (and the perpetrators).
- normalized databases.
- traceability.
- note that my memo system is an attempt at normalized knowledge management.
Yes, I think, within reason. This was anathema in the space industry, and I can see their point (admitting to the people who pay your company that you were wrong, incompetent, and a waste of money; and confusing everyone by putting "wrong" ideas in their heads, or at the least allow them to read stuff that they should not spend time on). But I beg to disagree. Corrected mistakes are more instructive than lucky hits, and help people avoid making the same mistake.
My first job was at an aircraft factory, where it was my job to collect, handle and disseminate flight test data. We are talking 10,000 parameters of in-flight measurement, organized per flight, and then per test run (typically 10 to 60 seconds), and then per parameter (each typically with its own calibration data), all to be made available in a few hours after the test flight had landed. I thought I had seen it all in terms of permanently traceable data storage and retrieval.
Upon joining the $ 1 billion one-shot ISO satellite project, I soon learned that the space industry takes "configuration control" to the next level. Although appalled at first by the bureuacracy involved in maintaining a sacred design document tree including a complete history of even the tiniest changes, I soon found that this was the only way to manage the decision tree in such a complex "first time right" project.
The configuration control experience at Space left an indelible mark on me, and it is probably the reason why I organized all my thinking into an elaborate personal memo system, from which this website is a tiny spin-off.
I'm sure I have gone overboard on the time I spent on maintaining my own documentation over the years, but I do believe that without it, I would have spent as much time trying to retrieve lost information or repeating old mistakes. I guess it's largely a matter of temperament. I just like to make slow, but steady progress.