One of the main reasons why it’s difficult to troubleshoot problems in complex environments is lack of documentation. This post is long overdue because it was inspired a while ago by Dan Fink’s post (again). Be sure to read it first.

From time to time, I see situations when one person leaving the company leads to a disaster that puts in danger the whole existence of multimillion businesses. Why? Because there is nobody else who knows how system works and how all its moving parts are connected. How does that happen? No knowledge sharing.

The most universal method of knowledge transfer is documenting stuff and I would argue that, most of the times, one hour of documenting brings to a business more than two hours developing new functionality in PL/SQL.

Actually, there might not even exist such know-it-all-wizard. Complex systems are not created in a matter of days or even months. Often, extreme complexity is the result of years or even decades of evolution. All those workarounds, bug fixes, re-designs, pressing deadlines — those friends of entropy win time from documentation tasks.

How many times it happened to you:
- Why are there forty-two underscore parameters in every new database you create?
- Because it’s our standard.
- Can I look at it?
- {proudly} Yes. It’s in every init.ora file we have!

Or another variation:

- Can I look at it?
- {proudly} Yes. Here is 42 pages document how to create new database:

Performance boosters:


and etc…

- When was this document started?
- {proudly} Oh! Forty-two years ago!
- When was it last time updated?
- Eh… I’m actually quite new here…

What are the reasons behind missing or inadequate documentation?
Just to name a few:
- It’s boring.
- We have no time for it now and it’s not a priority (it never will be) — we still missing couple reports.
- I’ll do it tomorrow.
- Project manager is not paid by number of pages of documentation produced (not that this is a good idea either).
- I’m the only one who knows this system — they must be idiots to get rid of me.
I’m sure readers can name plenty of others and please feel free to do so.

Sometimes you can avoid complexity and make systems simple by design but in real life it’s more often when you need to deal with complexity. Paraphrasing Dan — documentation will not make systems simpler but will reduce perceived complexity.

If you’ve got to support and/or develop a complex system — how do you make sure you dedicate enough time to documentation? What are the strategies to keep it up to date? How to explain the importance of documenting to the management? If you have anything to say — the comment box is at the bottom of this screen — use it. If you think that you have enough to say for the blog post — just let me know.

Find more about  , , ,

7 Responses to “Avoiding Guesswork in Complex Environments”  

  1. 1 Christian Antognini


  2. 2 Frits Hoogland

    Good points Alex! Exactly the last situation I was partly involved with.

    In my experience, setting up a successful system, either complex or simple, requires a systematic approach from the start. And, which cannot be said enough, simplicity!

    A database needs to be adjusted as little as possible. Sourcecode needs to be version controlled, Even in very small simple environments. Production environments need software releases, explicitly tagged from the version control. Not a few files to be fiddled with in order to make it work again.

    Happy easter!

  3. 3 Narendra


    Nice thoughts.
    I agree completely.
    Your last reason is very important. People, many times, do not share knowledge, due to insecurity. Sounds funny, but it is a fact. They (probably) get “married” to the company and whenever they are acknowledged as S.P.O.C. (Single Point-Of-Contact), it soothes their ego.
    Such people will do everything they can, to sabotage any plans to prepare / update documents.

  4. 4 Aleksey Tsalolikhin

    Documentation is very important and under-used in today’s IT shops.

    To help remedy that, here’s my recent post on system administrator site documentation with docbook, and a few “intro to docbook” links that were helpful to me.

  5. 5 Alex Gorbachev

    Good point. Thanks Aleksey.
    The biggest difficulty I found is to motivate engineers to write docs AND keep them up to date.

  6. 6 Graham Oakes

    Excellent points Alex.

    I would also add that writing the documentation is only useful if you ensure it’s properly distributed or can be easily found. Finding the correct location for the documentation and ensuring that it’s organised in such a way to make retrieval easy is not as simple as it could be in a large company(at least that’s what I’m observing where I work atm).

  7. 7 Alex Gorbachev

    Thanks Graham. I might be biased or just used to Oracle Database documentation style but after comparing Oracle database documentation with documentation for the products that Oracle acquired — I can only attest to your point. The ability to navigate quickly and effectively within a large documentation volumes is a must.

    But it’s not only the documents creators who should invest efforts, readers must invest enough to be familiar with organization and knowledge domain. Even when search functionality works great, you would still need to know what to search for.

    Thanks again for the feedback.

Leave a Reply

  • Bitchslap teen anal
  • Beyonce naked video
  • ebony pornstar fuck
  • teens begging for sex
  • hot young black girls
  • free streaming latina porn videos
  • free porn lesbian dildo tube
  • free online orgasmic porn videos
  • dick tracy erotic
  • hairy little girl porn video
  • free amatuer virgin sex videos
  • shiny bikini pictures
  • free porn women and couples
  • wet shaved teen pussy
  • plus size petite woman clothing knoxville
  • young girls suck tit
  • dailymotion topless girlfriend in car
  • sexy brenda song ass
  • free forced fuck picture
  • sexy teen pheonix
  • Hot teen fisting
  • girl downblouse pics
  • pussy egg breakfast
  • free videos of public flashing
  • free videos young teen fucking
  • fuck my younger sister
  • amature mature xxx pics
  • marie louise anal video
  • mature fucking pussy
  • sleeping girl gets fucked xxx nude
  • biggest dick in porn pics
  • database of pussy
  • south american girls who fuck
  • pussy brazil tv oops
  • lesbian sex anal
  • african tribe sex video
  • Maria bello nude videos
  • Weird things in vaginas
  • boat party nude girls video hitchhiker
  • free bobbi bliss deepthroat black dick
  • xxx young legal pussy
  • group sex stories husband and wife
  • weing day sex movies free
  • porn x videos
  • sex teen galleries
  • ft tall asian pussy
  • rocco fuck black girl pics
  • pics mature slut blowjobs
  • girls drinking pee galleries
  • how do i ass fuck mygirlfriend
  • tube teen sexweb
  • thick black movies
  • girl playing with tits black
  • free movie sex thumbs
  • uncut cheating wife videos
  • litle girls pictures
  • young mexican grils having sex
  • ashlee simpson nipple and pussy slips
  • gal pantyhose show hairy pussy
  • swedish girls hairy pussy
  • aspen teen pussy
  • star wars picture of alien girls
  • two young girls dressing up tube
  • video black bear camp nick
  • watch megan fox sex video
  • naked picture of dylan mcdermott
  • petite redhead orgy
  • Lesbian sucking videos
  • sex videos public
  • comedic monologues for young girls
  • free ebony girls videos
  • sexy blonde getting naked
  • xxx hardcore fetish erotica
  • famous pornstars anal pics
  • housewives nude mature
  • pretty asian video
  • young babes sucking
  • young sexy girl videos
  • tied up for sex video
  • swinging wife pictures sex
  • lsg nipple pussy
  • normal teen boobs pics
  • mature exhibitionism video
  • kim young wook korean singer
  • how to turn wife
  • English girl nude
  • black and white pictures of tamahome
  • Sex porn xxx video
  • drunk naked babes at party
  • busty girl eats pussy
  • couples seeking teens scene
  • black women white men anal
  • free video blowjob cock
  • big boobs videos solo
  • xxx sex old mature pichunter
  • big hard black dick
  • mature hairy pussy videos
  • moms amateur pics
  • naked male movie stars free pics
  • shemale rubbing dicks
  • Actress sex movies
  • college girls naked video free
  • Office girls xxx
  • interracial movie thumbs maturetoilet
  • amateur exhibitionist gallery
  • ashley tisdale porn videos
  • Hot chubby pussy
  • micks dick porn
  • free sex stories mature sluts
  • mothe likes to fuck younge boys
  • little cummed pussy
  • mature sexy granny tube
  • hot mfff sex pics
  • young nude russain girls
  • black teen keez
  • huge pussy swallows can
  • free shemale porn mpg galleries
  • young teen porn tibe
  • japanese babes and big boobs
  • erotic sex fantasy art
  • anal movies john holmes
  • white wives dating black men
  • jessie young leonard black holocaust museum
  • tall brunette video fuck
  • Girls next door nude scenes
  • wifey blonde handyman tool
  • wife slut galleries
  • Small girls nude pic
  • picture close up pussy
  • free housewife video fuck
  • asian xxx school
  • younger babes gallerie
  • freeanal sex videos
  • Girls sex sounds
  • sex invoving teens
  • indain shaved bald girl
  • horny black teen lesbians
  • free porn movies girl next door
  • husband leaving wife at airport pictures
  • naked jennifer lopez butt pictures
  • first time girl sex
  • amateur wife slow fuck american
  • razor for shaving pussy
  • ass backward sex
  • Sexy wow girl
  • young er teen porn pics
  • Iranian sex pictures
  • anal movie gallaries
  • Desi nude girls
  • sexy naibor girl porn
  • multiple oral sex
  • hot young girls getting banged
  • nonnude young teens in bras
  • caning porn video
  • amateur porn server
  • wife in a bikini
  • topless sport videos
  • customs officer lesbian porn
  • mature german family sex videos
  • hot naked teen asian girls
  • amature black couples homegrown vids
  • Lita nude pictures
  • free sex stores and pics
  • completely free girlfriend porn
  • young girl bath
  • Sara stone threesome
  • jordan capri cherry petite new
  • black dick deepthroat
  • very painful anal fuck girls
  • video trailers free girls
  • sexy ass tug job
  • fresh teen facial
  • free mature mom boy video gallery
  • freefemale orgasm video
  • free upload amateur sex video
  • young amateur bj galleries
  • pictures of full size naked women
  • free pics mature sluts getting beat
  • japan skirts with butt out
  • best amatuer fuck videos
  • Nylon sex porn
  • naughty babes movies
  • free porn full video big clit
  • ebony xxxx movies
  • blonde teen fucked hard
  • party girls swallowing
  • xxx vids vegas girls
  • girls having babies very young
  • best lesbian sex ever
  • light skin naked black girls
  • couple seduce slutty young girl
  • hot transvestite fuck girl
  • karen young naked
  • Sexy balloon girls
  • naked girls in sauna
  • Xxx pakistani sex
  • lots of dicks in girl
  • loving couples porno
  • video porn casero
  • home made movies bbw anal
  • sexy younger teen girls naked videos
  • Latinas teen nude
  • black girls squits
  • mature men pics
  • little girl loves sex true stories
  • Sexy body boobs
  • sex between girls and guys
  • see old black girl videos view
  • ultimate anal sex for men
  • free pussy worship
  • amateur free teen porn
  • girls using sex machines and squirt
  • Super sexy girls in bikinis
  • mature latina videos nude
  • wife slave girl initiation
  • wife revenge suck humiliated degraded sick
  • girls getting anal fingering
  • yellow bouquet with black party dress
  • naked mile video
  • Naked chinese ladies
  • black teen models by mail
  • hot butt fucking sex
  • emergency break in pussy
  • Jennifer lopez ass shaking
  • black panther sex with toon
  • latin girls black dick
  • blacks fuck virgins
  • glmour babes pussy
  • nude videos mature
  • home make sex movies
  • ass rump pussy
  • free hot sexy wet nude babes
  • Not another teen movie cast
  • black chicks with dildo
  • Sex models nude
  • tiffany holiday anal flash videos
  • Amazing oral sex
  • mature sex amateur free
  • monster prick penetrating her virgin pussy
  • shannon tweed hairy pussy
  • georgia jones nude fuck videos
  • free o lesbian porn xxx
  • real amature orgasm pics
  • young girl streaming porn
  • naked fat videos
  • Beautiful girl sex
  • oil porn video
  • crystal klein bikini riot videos
  • free beauty and the senior pics
  • indian homemade videos by couples
  • the best rated porn video free
  • pornstar blowjob videos
  • wet hentai pussy
  • nonnude young sex pics
  • Bump on outside of vagina
  • girls group sex at swimming pool
  • pics of best tight ass
  • hot teenage blonde with huge tits
  • hot young blonde sex pics
  • wigs by pierre petite feather lite
  • topless bikini gallery
  • free xxx movies hardcore teacher
  • amateur wives handjobs
  • white chicks fucking nigger dick
  • sexy teen leashes
  • Hot girls in t shirts
  • dragon naked girl
  • sex lesbo babes
  • skinny erotic nude teen sex
  • free anal fisting galleries
  • wife gives blow jobs video
  • girls pictures allergies
  • lesbian orgies porn
  • sex long free video
  • sexy russian brunette teen masha
  • girls kissing on a be video
  • beach pics nude couples
  • teen girl swimsuit pictures
  • little teens forced to fuck
  • sweet little young girls
  • cuties teen amateur models
  • malika b grade movie sex
  • cancan girls pics
  • girl shit on dick porn
  • free fuck my wife gangbang stories
  • free xxx ebony dvds
  • ashile simpsons pussy
  • lesbian sex vocal online
  • Girls make sex
  • black fat ass brazilian diana
  • xvideos blake rose interracial anal
  • young asian teenagers
  • womans point of view sex video
  • xxxx megga boobs lactaing you tube
  • girls with sloppy pussys
  • amatuer video amateur men bi
  • very young non nude xxx
  • guys naked wrestling for sex
  • stolen videos amature porn
  • Cum in feet
  • pussy toilet close up
  • big tit mature full vids
  • Interracial cuckold humiliation
  • really young girls in thongs
  • all girl sex dance party
  • little sexy nude girls
  • pussy grinding pussy
  • sex movie long free
  • free ebony women solo pictures
  • free interracial wife movies
  • free whippped ass movies
  • christina agulera pussy
  • mom and teen free pictures
  • fucking hot girl pics
  • preparation sex anal
  • Jenifer lopez sex scenes
  • black boobs hairy legs black
  • pornstar terri conroy pics
  • teen blonde cunt
  • florida sex party
  • metacafe naked nude sex
  • string bikini girl
  • Sex scene from hollywood
  • teach young girl stories
  • bubble butt bikini pics
  • Lesbian deep ass licking
  • Fuck sex girls
  • naked girls southfield
  • jessica jaymes hardcore videos
  • sex in the city pics
  • sexy nude young tiny tity teens
  • hot celeb sex movies
  • softcore free long videos
  • girl next door video cam
  • neve cambell naked pics
  • average girls try porn
  • girl having her first orgasm squrting
  • sexy naked men galleries
  • boobs press video
  • petite dainty nude teen
  • sexy picsof girls for my space
  • young virgin pussy in there teens
  • mature lesbian big tits
  • shemale cum female xvideos
  • best amateur nude pictures
  • wife surprise gang bang
  • sex post amateur video college dorm
  • teen puffy real blonde pubes
  • brutal asian sex
  • photo wife naked hubby
  • chocolate milk white girl pussy
  • adult movie hal asian cable stars
  • safe to have anal sex
  • pictures of hungarian ethnic costumes
  • black amateur freaks
  • girlfriends butt smells like poop
  • anal ametuers ebony
  • porn interratial couples orgy movies free
  • topless bikini beach pics
  • wife murders girlfriend
  • Indian tit sex
  • granny i sex video
  • big ass in panties galleries
  • free interracial lesbian domination
  • sologirl petite tits
  • oedipus sex xxx
  • plump blonde pussy thumbs
  • free video two girls
  • little cute black girls
  • hardcore teen first time
  • free crystal heart fuck videos
  • my first black dick
  • pussy ass fisting porn
  • lesbian bondage stories anal sex
  • erotic sex stories sister threesome
  • picture ideas for teens
  • marilyn monroe porn pictures
  • free blonde amine porn videos
  • free mature creampie galleries
  • xxx asian anal whore
  • anne hathaway sex video nude
  • nice round tits and pussy
  • third sex free sex pics
  • wife sex vaion video
  • she showed me herr pussy
  • amateur nipple pics
  • japan upskirt porn
  • free lesbian sex pic gallaries
  • adult black home videos
  • Hot milky tits
  • naked girl camera
  • youmg girls threesome
  • free black porn in jordans
  • Kisses her ass
  • crazy big pussy vids
  • japan movie sex
  • erotic sex stories greek
  • porn hairless young legal male teen
  • beautiful stretched pussy
  • guy eating a girl out video