One of the main reasons why it’s difficult to troubleshoot problems in complex environments is lack of documentation. This post is long overdue because it was inspired a while ago by Dan Fink’s post (again). Be sure to read it first.

From time to time, I see situations when one person leaving the company leads to a disaster that puts in danger the whole existence of multimillion businesses. Why? Because there is nobody else who knows how system works and how all its moving parts are connected. How does that happen? No knowledge sharing.

The most universal method of knowledge transfer is documenting stuff and I would argue that, most of the times, one hour of documenting brings to a business more than two hours developing new functionality in PL/SQL.

Actually, there might not even exist such know-it-all-wizard. Complex systems are not created in a matter of days or even months. Often, extreme complexity is the result of years or even decades of evolution. All those workarounds, bug fixes, re-designs, pressing deadlines — those friends of entropy win time from documentation tasks.

How many times it happened to you:
- Why are there forty-two underscore parameters in every new database you create?
- Because it’s our standard.
- Can I look at it?
- {proudly} Yes. It’s in every init.ora file we have!

Or another variation:

- Can I look at it?
- {proudly} Yes. Here is 42 pages document how to create new database:

Performance boosters:
_db_aging_touch_time=…
_db_aging_stay_count=…
_db_writer_chunk_writes=…
_discrete_transactions_enabled=…

Bugs:
_kgl_latch_count=…
_kgl_bucket_count=…
_lm_rcfg_timeout=…

and etc…

- When was this document started?
- {proudly} Oh! Forty-two years ago!
- When was it last time updated?
- Eh… I’m actually quite new here…

What are the reasons behind missing or inadequate documentation?
Just to name a few:
- It’s boring.
- We have no time for it now and it’s not a priority (it never will be) — we still missing couple reports.
- I’ll do it tomorrow.
- Project manager is not paid by number of pages of documentation produced (not that this is a good idea either).
- I’m the only one who knows this system — they must be idiots to get rid of me.
I’m sure readers can name plenty of others and please feel free to do so.

Sometimes you can avoid complexity and make systems simple by design but in real life it’s more often when you need to deal with complexity. Paraphrasing Dan — documentation will not make systems simpler but will reduce perceived complexity.

If you’ve got to support and/or develop a complex system — how do you make sure you dedicate enough time to documentation? What are the strategies to keep it up to date? How to explain the importance of documenting to the management? If you have anything to say — the comment box is at the bottom of this screen — use it. If you think that you have enough to say for the blog post — just let me know.

Find more about  , , ,

7 Responses to “Avoiding Guesswork in Complex Environments”  

  1. 1 Christian Antognini

    Amen!

  2. 2 Frits Hoogland

    Good points Alex! Exactly the last situation I was partly involved with.

    In my experience, setting up a successful system, either complex or simple, requires a systematic approach from the start. And, which cannot be said enough, simplicity!

    A database needs to be adjusted as little as possible. Sourcecode needs to be version controlled, Even in very small simple environments. Production environments need software releases, explicitly tagged from the version control. Not a few files to be fiddled with in order to make it work again.

    Happy easter!

  3. 3 Narendra

    Alex,

    Nice thoughts.
    I agree completely.
    Your last reason is very important. People, many times, do not share knowledge, due to insecurity. Sounds funny, but it is a fact. They (probably) get “married” to the company and whenever they are acknowledged as S.P.O.C. (Single Point-Of-Contact), it soothes their ego.
    Such people will do everything they can, to sabotage any plans to prepare / update documents.

  4. 4 Aleksey Tsalolikhin

    Documentation is very important and under-used in today’s IT shops.

    To help remedy that, here’s my recent post on system administrator site documentation with docbook, and a few “intro to docbook” links that were helpful to me.

  5. 5 Alex Gorbachev

    Good point. Thanks Aleksey.
    The biggest difficulty I found is to motivate engineers to write docs AND keep them up to date.

  6. 6 Graham Oakes

    Excellent points Alex.

    I would also add that writing the documentation is only useful if you ensure it’s properly distributed or can be easily found. Finding the correct location for the documentation and ensuring that it’s organised in such a way to make retrieval easy is not as simple as it could be in a large company(at least that’s what I’m observing where I work atm).

  7. 7 Alex Gorbachev

    Thanks Graham. I might be biased or just used to Oracle Database documentation style but after comparing Oracle database documentation with documentation for the products that Oracle acquired — I can only attest to your point. The ability to navigate quickly and effectively within a large documentation volumes is a must.

    But it’s not only the documents creators who should invest efforts, readers must invest enough to be familiar with organization and knowledge domain. Even when search functionality works great, you would still need to know what to search for.

    Thanks again for the feedback.

Leave a Reply


  • extra tight pink pussy
  • black men breeding my wife
  • teen asian london escorts
  • family sex videos free
  • chauvinistic men sex face fuck
  • Naked slave girls
  • french girl threesome
  • teen young ass
  • www sweed sex girls
  • milf and anal video
  • sex brown ass
  • all sex video upload
  • World beautiful pussy
  • Fucking with big tits
  • fuck teeny boppers
  • young blonde pussys
  • free lesbian piss videos
  • wife free sex
  • olympics sex contest video
  • irish school girls naked
  • college girl naked with guys picture
  • fucking pussy outdoor
  • free hardcore mature women pics
  • Japan hot sexs
  • lopez anal video
  • mature hairy black pussy pictures
  • free xxx lesbian video ass
  • sexy foot worship girls
  • met art petite
  • Naked booty shaking videos
  • tiny teen girls fuck football
  • itaewon sex girls
  • german star photo gallery
  • amateur videos red heads interacial
  • xxx hardcore movies
  • nude beach sex party
  • hardcore hot sex video
  • Small tits strip
  • blueyed cass shows her pussy
  • free movies with sex
  • sexy blonde bukkake
  • wife wrestling movies
  • teen busty blonde dped
  • free dark littel teen bitch
  • online hentai sex videos
  • Nude amateurs girls
  • sexy hot korean young girls video
  • video hardcore redheads
  • free teen jpeg anal
  • huge black white teen
  • kayo petite capri pants
  • Latina pussy and ass
  • couples sex on beach
  • interracial panther party
  • neil young rust never sleeps video
  • breasts ass pussy pictures
  • black lesbian fighting megs
  • free ass clit vids
  • young teen models shaved panties
  • Anal cum eating
  • black booty pussy eating
  • Sexy girl pin up
  • messy blowjob videos
  • free asian homemade sex videos
  • free video of hanna star
  • no registration free blowjob videos
  • music for teenage girls favorite songs
  • amateur jerk off video
  • old womens hanging breasts free galleries
  • naked runner video
  • sexy chubby mature babes video
  • movies hardcore
  • gothic woman sex videos
  • denver co shaved pussy
  • anal sex lining torn
  • russian girl orgasm video
  • Boleyn girl sex scene
  • hot blonde school girl
  • young teen chubby cum
  • black interracial tube
  • Chubby natural tits
  • nina hartley facial free pics
  • when wives dont like sex
  • pictures and daughter
  • Asian black lesbian
  • European nude teen models
  • teen sex old dude
  • hot naked girls having sex free
  • Hot tiny teen
  • redhead galleries hardcore
  • free wife anal sex stories
  • mature sex orgys
  • free videos girls cherrys popped
  • best first time anal virgin video
  • asian tight anal
  • adelaide amateur girls
  • girls flashing tits video
  • amatuer wife fucking first blacks
  • wife sexy ass
  • african ass cunts pussy
  • ebony free sex video cum
  • Movies teen nude
  • Colleen shannon nude pics
  • Teen sexy pic
  • Old nude pussy
  • black white pussy cock
  • cute topless galleries
  • ebony granyy sexvideos
  • teen slut black
  • sexy naked black teens
  • world teen girls having sex
  • bbw ebony lesbians tube
  • download and free hardcore xxx
  • pick up teens for sex
  • the art of anal sex
  • girls nike red black cleat
  • post operative transsexual orgasm
  • girls in uniforms free pics
  • tiny teens small tits movie
  • little girls naked haveing sex
  • free ass pic gallery
  • young naked girls gallery
  • alex reid killer bitch movie stream
  • super hot shemale cumshot videos
  • webcam metallica girl poynter
  • great amateur fuck videos
  • sex ass huge
  • First nude photo shoot
  • blonde teen tutor slutload
  • banned teen sex pics
  • ass to mouth asian video
  • watch big dick bitch video free
  • hardcore bear
  • cy young lee korean actress
  • fresh young teen nipples
  • black bike week sex
  • free videos of ebony women masterbating
  • whore anal pics
  • free naked teenie
  • wife wants my dick bigger
  • amateur movies videos
  • free forced hardcore bride pics
  • teens bikini candids
  • movies of sexy black shemales fucking
  • teen xxx websites
  • tits jugs melons s movies
  • kimberly holland nude shaved pussy
  • My sweet six teen
  • dicks on big tits
  • teens wanting sex
  • mature sex mpegs
  • sex literature pictures
  • Hairy sexy pussy
  • asian xxx list
  • pussy pump stars
  • nude sex adult video
  • young sister fuck brother
  • naked asian girl videos
  • Lesbien sex pictures
  • Ebony cum in mouth
  • slick swolen pussy
  • sex stories free wife
  • eats girl pussy story
  • fuck women sex pictures
  • girls sex underwater
  • sex xxx beautiful
  • femme petite ue fellation
  • mature nylon feet galleries
  • japanese nude girls strip show
  • paula abdual naked pics
  • little young teen nude
  • amateur lesbians in public
  • erotic xxx story by jane parks
  • japan student video streaming
  • young japanese huge tits video
  • free nude asian girls video
  • sex ass pising video
  • Sexy girls cum
  • video prevews
  • japanese woman blowjob on bus
  • Gta sexy girls
  • Big boob bangeroo
  • national anal sex association
  • armenian girl for sex
  • orlds largest pussy
  • virgin wet pussy
  • mini tits gallery
  • extreme party hardcore
  • free big butt video thumbnails
  • amateur interracial sex free video
  • hardcore bdsm tubes
  • wife makes husband suck another man
  • free handjob videos online
  • keez big tits lesbians
  • shemale on girl
  • Naked lady guitar
  • sexy girls of college
  • girls home sex
  • video of handjob
  • bikini gallery micro
  • farmer girl pics
  • max hardcore tube two teens
  • redtube french girls xxx
  • girls have sex in shop
  • dy young girls sex video
  • black girls interracial
  • pinoy pride black bikini briefs
  • eat my big mature pussy
  • topless photos erotic teen art
  • bikini dance videos
  • Hot girls in brazil
  • sleeping teen fuck movies
  • mature asian dvds
  • Large ladies naked
  • hardcore blk on wht
  • facial movies gallery
  • Tila tequila lesbian video
  • athletic girl sex
  • Hot girls only
  • so good in my pussy
  • dailymotion sexy girl strip naked
  • bbw freee sex videos
  • girls xxx blackdick in teens
  • pamela anderson hardcore pictures
  • girls gone wild louisiana free video
  • virgin wife impregnated by black
  • Nude indian girl sex
  • young sex babes
  • teen sex great barrington ma
  • Big black booty pussy
  • nice tits latina sex videos
  • nude chubby mature women picture galleries