It is every software engineer’s nightmare – and it had happened to me once ten or so years ago. I was uploading my latest code, a lot of it, onto a server to begin to compile it, and at the same time I needed to update my local working folder structure with the other stuff our team did, but the server crashed while both tasks were in progress.
I remember sitting there slowly realising what has been lost, my heart thumping away, all them weeks of slogging away gone down the ether-drain…… Luckily we had a talented system admin on site, and we managed to look through archives to get back as much stuff as we could, and I only needed to redo two or three days’ worth of work – which wasn’t too bad in the end!
Since then, we’ve replaced Windows servers with Linux machines, so they’re far, far more reliable, but better safer than sorry; I’ve always backed stuff up properly, and only do an upload or update and not both at the same time (as I can’t see what the server is actually doing!) 🙂
A couple of years back, the European Space Agency were planning a partnership with NASA for a series of instruments to send to Mars, with the principal aim of finding out if there’s any sign of life, but with lots of other experiments on board too.
We were lucky enough at Oxford to be working on four of the instruments on the lander, and were being asked to provide input on a fifth. Then the financial crash happened and the mission was cancelled.
Suddenly, we went from having lots of work and plenty of jobs to no work at all. That’s was pretty depressing.
I’m quite lucky nothing too bad has happened to me. I’ve embarrassed myself by sending out a report with a silly mistake, but everyone has done that at some point, surely! So I don’t let it bother me and just try to learn from mistakes.
Not recently but as a young engineer I remember badly misunderstanding a client once about what we had to do and by when. The guy was pretty upset saying my action was going to cost money, cause delay and be a bad mark. He was nastier than a nasty thing. But ever since I feel to have worked pretty hard to be clear about what people want and to get issues on the table sooner rather than later.