One of the main reasons why it’s difficult to troubleshoot problems in complex environments is lack of documentation. This post is long overdue because it was inspired a while ago by Dan Fink’s post (again). Be sure to read it first.

From time to time, I see situations when one person leaving the company leads to a disaster that puts in danger the whole existence of multimillion businesses. Why? Because there is nobody else who knows how system works and how all its moving parts are connected. How does that happen? No knowledge sharing.

The most universal method of knowledge transfer is documenting stuff and I would argue that, most of the times, one hour of documenting brings to a business more than two hours developing new functionality in PL/SQL.

Actually, there might not even exist such know-it-all-wizard. Complex systems are not created in a matter of days or even months. Often, extreme complexity is the result of years or even decades of evolution. All those workarounds, bug fixes, re-designs, pressing deadlines — those friends of entropy win time from documentation tasks.

How many times it happened to you:
- Why are there forty-two underscore parameters in every new database you create?
- Because it’s our standard.
- Can I look at it?
- {proudly} Yes. It’s in every init.ora file we have!

Or another variation:

- Can I look at it?
- {proudly} Yes. Here is 42 pages document how to create new database:

Performance boosters:
_db_aging_touch_time=…
_db_aging_stay_count=…
_db_writer_chunk_writes=…
_discrete_transactions_enabled=…

Bugs:
_kgl_latch_count=…
_kgl_bucket_count=…
_lm_rcfg_timeout=…

and etc…

- When was this document started?
- {proudly} Oh! Forty-two years ago!
- When was it last time updated?
- Eh… I’m actually quite new here…

What are the reasons behind missing or inadequate documentation?
Just to name a few:
- It’s boring.
- We have no time for it now and it’s not a priority (it never will be) — we still missing couple reports.
- I’ll do it tomorrow.
- Project manager is not paid by number of pages of documentation produced (not that this is a good idea either).
- I’m the only one who knows this system — they must be idiots to get rid of me.
I’m sure readers can name plenty of others and please feel free to do so.

Sometimes you can avoid complexity and make systems simple by design but in real life it’s more often when you need to deal with complexity. Paraphrasing Dan — documentation will not make systems simpler but will reduce perceived complexity.

If you’ve got to support and/or develop a complex system — how do you make sure you dedicate enough time to documentation? What are the strategies to keep it up to date? How to explain the importance of documenting to the management? If you have anything to say — the comment box is at the bottom of this screen — use it. If you think that you have enough to say for the blog post — just let me know.

Find more about  , , ,

7 Responses to “Avoiding Guesswork in Complex Environments”  

  1. 1 Christian Antognini

    Amen!

  2. 2 Frits Hoogland

    Good points Alex! Exactly the last situation I was partly involved with.

    In my experience, setting up a successful system, either complex or simple, requires a systematic approach from the start. And, which cannot be said enough, simplicity!

    A database needs to be adjusted as little as possible. Sourcecode needs to be version controlled, Even in very small simple environments. Production environments need software releases, explicitly tagged from the version control. Not a few files to be fiddled with in order to make it work again.

    Happy easter!

  3. 3 Narendra

    Alex,

    Nice thoughts.
    I agree completely.
    Your last reason is very important. People, many times, do not share knowledge, due to insecurity. Sounds funny, but it is a fact. They (probably) get “married” to the company and whenever they are acknowledged as S.P.O.C. (Single Point-Of-Contact), it soothes their ego.
    Such people will do everything they can, to sabotage any plans to prepare / update documents.

  4. 4 Aleksey Tsalolikhin

    Documentation is very important and under-used in today’s IT shops.

    To help remedy that, here’s my recent post on system administrator site documentation with docbook, and a few “intro to docbook” links that were helpful to me.

  5. 5 Alex Gorbachev

    Good point. Thanks Aleksey.
    The biggest difficulty I found is to motivate engineers to write docs AND keep them up to date.

  6. 6 Graham Oakes

    Excellent points Alex.

    I would also add that writing the documentation is only useful if you ensure it’s properly distributed or can be easily found. Finding the correct location for the documentation and ensuring that it’s organised in such a way to make retrieval easy is not as simple as it could be in a large company(at least that’s what I’m observing where I work atm).

  7. 7 Alex Gorbachev

    Thanks Graham. I might be biased or just used to Oracle Database documentation style but after comparing Oracle database documentation with documentation for the products that Oracle acquired — I can only attest to your point. The ability to navigate quickly and effectively within a large documentation volumes is a must.

    But it’s not only the documents creators who should invest efforts, readers must invest enough to be familiar with organization and knowledge domain. Even when search functionality works great, you would still need to know what to search for.

    Thanks again for the feedback.

Leave a Reply


  • samoan naked women pictures
  • butt movie galleries
  • sexy outfits for shorter husky girl
  • mature daily vids
  • eva black listing
  • sexual teachers and young girls
  • naked pictures of dareyl worrely
  • mature rough japanese sex
  • english patient sex scene video
  • hairy flat chested nude girl galleries
  • Deep throat oral sex
  • tiny asian anal
  • frist time black anal
  • sex parlor pics
  • little girl picture attached to email
  • free younge
  • asian stockings heels thumbnails gallery
  • free girl sex fuck movies
  • jennifer white naked pictures
  • stripping mature videos
  • best babes stripping videos
  • sexy string bikini naked
  • wife trys black
  • emo girl web cam boobs
  • asian hardcore gangbang
  • Naked latina models
  • Hot milf tits
  • erotic valentine gifts for your wife
  • go karts for teens pictures
  • anal blonde whore
  • free minutes movie monster
  • xxx legs and ass
  • enema girls young
  • girlfriend jess naked
  • deutsche amateur sex filme
  • hot chubby sexy girls
  • shaved wey pussy
  • Hard on sex
  • oral sex tips for s
  • Heather graham naked pics
  • surfers amateur videos
  • jill party girl
  • brutal pussy screaming
  • Emma watson nude photos
  • fake sarah palin sex pics
  • teen hardcore vagina
  • public sex videos in bus
  • sex toys for women erotic books
  • suck black men
  • amature girls big boobs stripping
  • candice nicole video
  • teen amatuer videos
  • Tamil sexy college girls
  • black girl big boobs blue paint
  • disgusting old bitch sex
  • her only movie
  • big tits anal milfs
  • mofosex young boys video
  • real crazy sex videos
  • teen ebony riding cock
  • free pics of lengerie sex
  • real young shemale boys
  • Teen tits movie
  • sexy free videos
  • fuck pics moms
  • Teen pregnancy syptoms
  • softcore pictures gallery
  • young couple movie
  • free roleplaying amateur
  • step moms that fuck videos
  • naked black ass mini
  • Sexy girls in hawaii
  • hot young lesbian girls
  • Indian village girl nude
  • Big tit milking
  • major pussy penetration video
  • king reality gallery
  • lesbian fuck emo tribbing
  • Free teen pix
  • wife creampie black
  • mature sex mother son
  • blonde girl violin players
  • sexy girl pics
  • Pretty girl
  • mature on boy sex movie galleries
  • sexy asain girls
  • video schoolgirl tits forced
  • Paris hilton sex videos
  • las vegas sex teen prostitute
  • older naked men pictures
  • free girls riding cock movies
  • fucking pics of married couples
  • juicey black pussy
  • prettiest petite models
  • amateur free movie site
  • web teen sex slut
  • kira reed lesbian sex scene
  • Perfect tit sex
  • wife hairy orgasm creampie
  • nebule fuck young teen
  • Penis versus vagina
  • sexy asian lesbian skirt
  • group sex scenes
  • Porn tits videos
  • mom son video amateur
  • sexy blonde teen blowjob
  • anal sex interracial
  • piercing the clit while sex videos
  • mature men teenage gallery
  • oldyoung women movies
  • best boy video
  • free sex hardcore xxx vids
  • pregnant erotic sex
  • Sexy girls in sweatpants
  • japanese sex and drum video
  • ashley alexandra dupre sex video
  • lesbian orgy pics
  • amateur home group sex
  • heavy blackl ass
  • shaved pussy cum
  • cassie young fucked hardcore
  • hot girls spreading pussy
  • movies local mn
  • girls of ipanema naked
  • girl and pictures
  • british star pics
  • stream barely legal videos
  • spreads her pussy outdoors
  • indecent wife videos
  • korean lesbian school girls
  • mature women with boys sex videos
  • bikini black hair
  • asian couple in the movie crash
  • Beach girls breast
  • new teen cumshot videos
  • search engines soft video
  • butt sex big cock
  • Hot girl pranks
  • mature gaping old pussy
  • hairy pussy interracial
  • Vergin sex video
  • free webcam sex very young
  • ethnic sex video share
  • movies of young sexy feet
  • Kendra wilkinson nude picture
  • on my butt
  • blsck snd white pussy
  • girls showing ass photos
  • men fingering woman to climax videos
  • mature facial cum
  • very young naked little girls
  • teen ass fucking movies
  • long dick suck
  • Enema sex pics
  • tits and ass gallerys
  • sexy asian massage tube
  • sensual topless massage video
  • bugs bunny sex gallery
  • blonde tranny
  • naked teen men amateur
  • mature bra galleries
  • female athletic butt pics
  • orgasm prepubescent girls babies
  • choe movies
  • submissive mature moms suck sons dick
  • Giant pussy movies
  • free cum in ass videos
  • Water jet masturbation
  • wife having hard sex
  • voyeur naked posted free pics
  • nn teen girl pictures
  • movies naked wife
  • big black butts
  • transgender teen movies
  • bikini girls and chopin
  • bikini ebony sex
  • rollos big black boobs
  • hien video of wife
  • university girls naked videos
  • Korean girl naked
  • group sex elephant list
  • teen sex stat
  • free videos young amatuers fucking
  • bdsm erotic movies
  • old man pounding teen pussy
  • longest self suck dick video
  • glamour amateurs videos
  • kogal idol pussy
  • girl fuck knot hung
  • wet female pussy movies
  • greek amateur pics
  • mom daughter sex combo videos
  • eat cumshot video
  • somali fuck biggest ass ever xxx
  • free amateur love videos
  • naked primitives pics
  • sexual facial grimaces are
  • girl with big boobs
  • men eating cum from womans pussy
  • dy boy sex videos
  • free chinese fuck movies
  • tube sites with amateur orgy
  • teen boys hien shower pictures
  • indian s sex videos
  • black hoochie suck white
  • indian girls in sex free sites
  • very young naked afro girls
  • sexy petite brunette
  • big ass busty galleries
  • Teens riding cock
  • amateur coworker sexy candid video
  • aurora snow interracial anal videos
  • anal ass pounding
  • free videos of eating pussy
  • kate bosworth ass pics
  • monster anal sex black
  • girl jumping on dick