One of the main reasons why it’s difficult to troubleshoot problems in complex environments is lack of documentation. This post is long overdue because it was inspired a while ago by Dan Fink’s post (again). Be sure to read it first.

From time to time, I see situations when one person leaving the company leads to a disaster that puts in danger the whole existence of multimillion businesses. Why? Because there is nobody else who knows how system works and how all its moving parts are connected. How does that happen? No knowledge sharing.

The most universal method of knowledge transfer is documenting stuff and I would argue that, most of the times, one hour of documenting brings to a business more than two hours developing new functionality in PL/SQL.

Actually, there might not even exist such know-it-all-wizard. Complex systems are not created in a matter of days or even months. Often, extreme complexity is the result of years or even decades of evolution. All those workarounds, bug fixes, re-designs, pressing deadlines — those friends of entropy win time from documentation tasks.

How many times it happened to you:
- Why are there forty-two underscore parameters in every new database you create?
- Because it’s our standard.
- Can I look at it?
- {proudly} Yes. It’s in every init.ora file we have!

Or another variation:

- Can I look at it?
- {proudly} Yes. Here is 42 pages document how to create new database:

Performance boosters:
_db_aging_touch_time=…
_db_aging_stay_count=…
_db_writer_chunk_writes=…
_discrete_transactions_enabled=…

Bugs:
_kgl_latch_count=…
_kgl_bucket_count=…
_lm_rcfg_timeout=…

and etc…

- When was this document started?
- {proudly} Oh! Forty-two years ago!
- When was it last time updated?
- Eh… I’m actually quite new here…

What are the reasons behind missing or inadequate documentation?
Just to name a few:
- It’s boring.
- We have no time for it now and it’s not a priority (it never will be) — we still missing couple reports.
- I’ll do it tomorrow.
- Project manager is not paid by number of pages of documentation produced (not that this is a good idea either).
- I’m the only one who knows this system — they must be idiots to get rid of me.
I’m sure readers can name plenty of others and please feel free to do so.

Sometimes you can avoid complexity and make systems simple by design but in real life it’s more often when you need to deal with complexity. Paraphrasing Dan — documentation will not make systems simpler but will reduce perceived complexity.

If you’ve got to support and/or develop a complex system — how do you make sure you dedicate enough time to documentation? What are the strategies to keep it up to date? How to explain the importance of documenting to the management? If you have anything to say — the comment box is at the bottom of this screen — use it. If you think that you have enough to say for the blog post — just let me know.

Find more about  , , ,

7 Responses to “Avoiding Guesswork in Complex Environments”  

  1. 1 Christian Antognini

    Amen!

  2. 2 Frits Hoogland

    Good points Alex! Exactly the last situation I was partly involved with.

    In my experience, setting up a successful system, either complex or simple, requires a systematic approach from the start. And, which cannot be said enough, simplicity!

    A database needs to be adjusted as little as possible. Sourcecode needs to be version controlled, Even in very small simple environments. Production environments need software releases, explicitly tagged from the version control. Not a few files to be fiddled with in order to make it work again.

    Happy easter!

  3. 3 Narendra

    Alex,

    Nice thoughts.
    I agree completely.
    Your last reason is very important. People, many times, do not share knowledge, due to insecurity. Sounds funny, but it is a fact. They (probably) get “married” to the company and whenever they are acknowledged as S.P.O.C. (Single Point-Of-Contact), it soothes their ego.
    Such people will do everything they can, to sabotage any plans to prepare / update documents.

  4. 4 Aleksey Tsalolikhin

    Documentation is very important and under-used in today’s IT shops.

    To help remedy that, here’s my recent post on system administrator site documentation with docbook, and a few “intro to docbook” links that were helpful to me.

  5. 5 Alex Gorbachev

    Good point. Thanks Aleksey.
    The biggest difficulty I found is to motivate engineers to write docs AND keep them up to date.

  6. 6 Graham Oakes

    Excellent points Alex.

    I would also add that writing the documentation is only useful if you ensure it’s properly distributed or can be easily found. Finding the correct location for the documentation and ensuring that it’s organised in such a way to make retrieval easy is not as simple as it could be in a large company(at least that’s what I’m observing where I work atm).

  7. 7 Alex Gorbachev

    Thanks Graham. I might be biased or just used to Oracle Database documentation style but after comparing Oracle database documentation with documentation for the products that Oracle acquired — I can only attest to your point. The ability to navigate quickly and effectively within a large documentation volumes is a must.

    But it’s not only the documents creators who should invest efforts, readers must invest enough to be familiar with organization and knowledge domain. Even when search functionality works great, you would still need to know what to search for.

    Thanks again for the feedback.

Leave a Reply


  • asian on white
  • free brothers and sisters sex movies
  • young girl teen naked
  • blonde lesbian stars hardcore lesbian sex
  • european party
  • young girl natural tits
  • Scene girls xxx
  • hot girl hot tits
  • video streaming western classic retro
  • videos de amateur
  • Naked nepali girls
  • snowtrails bikini pics
  • asian nostril licking sex
  • hardcore teen school
  • story girls sex
  • teens pussy fucking oral sex scenes
  • teen girl cumshot
  • post girlfriend beach purple blurred topless
  • Girl fuck pics
  • free new sleeping sex movies
  • naked teen babby sitters
  • perfect pink teen pussy
  • teen does anal
  • black star cum
  • handjob teen photo gallery
  • naked sexy ladys
  • petite white sweaters
  • threesome ebony
  • girl pussy smell
  • sexy teen vaginas hot dominant
  • black studs sucking dick
  • matureworld yannis mature sex forum
  • new lindsay lohan sex pictures
  • anal sex portland
  • young teen pantyhose
  • sailor moon shemale videos
  • joke wife picture of boat
  • oral sex cum shots
  • young pussy up close videos
  • teen virgin fucked pics
  • totty teen girl poses
  • free gaint dildos movies
  • happy naked teens
  • cumshot tits pics
  • picture of pussy and dicks
  • hermaphrodites sex pics free
  • asian twink free video
  • hot sexy boobs sex pics
  • petite brunette masterbating
  • sex best friends girlfriend
  • sex stag party for women
  • pictures of beyonces butt
  • girlfriend suck dick
  • statistics on sexually active teens
  • Sharon stone sex scenes
  • skinny lesbian teens sex
  • teen sex young
  • kids bikini videos
  • erotic teen sex videos
  • teens scissoring video
  • Virginie ledoyen nude video
  • white girl cumshots
  • pregnacy with anal sex
  • Black ass master
  • free classic star video
  • str teen videom
  • stretch petite short pants
  • older women younger boys fuck
  • granny anal pics
  • Killing me softly movie sex
  • petite girls big toy
  • hot brunnet tits sexy
  • squeaky clean pussy
  • ebony girlfriend pics
  • Nude sex
  • free online crying fat movies
  • lesbian hentai rough sex
  • aphrodite orgasm videos
  • Nude picture of ash
  • girls you know going topless
  • tiger woos star girl friend
  • oral sex aiction
  • brazilian teen lesbian
  • real webcam sex full length
  • University hot girls
  • Japanes sex movie
  • Porn sex tits
  • rubbing tits video
  • Sexy iran girls
  • young girls nickers
  • hottest shemale pics
  • naked girls mountain hiking
  • handjob schoolgirl story
  • clasic erotic video collections
  • erotic story for girls
  • little girl naked pussy
  • free hardcore daily videos
  • Smoll girls sex
  • jennifer love hewitt hardcore movies
  • shemale on girl videos free
  • snapping pussy creampie
  • bbw asian tv
  • japan girl masterbation tube
  • free videos handjob
  • tight young teen movies
  • larry levine petite pants
  • Incest sex galleries
  • party naked pictures
  • girls to young for pics
  • first brunette asian anal
  • public flashing movies
  • busty curvey girl
  • naked pictures of cheerleaders
  • free suck my cock movies
  • babes blondes
  • passed out anal fuck
  • wife on wife sex video
  • japanese girls in sailor outfits
  • anal gaping
  • Nude indian grils
  • video young girl self fisting
  • tits boobs hardcore sex
  • free girls who squirt cum
  • domanate oral sex
  • erotic jailbait girl
  • free huge nude girl pics
  • girl homemade sex
  • homemde fuck videos
  • Russian teen whore
  • Angelina jlie sex scene
  • teen amature movies free
  • young petite hotties
  • free full sex videos megerotica
  • free caught having sex videos
  • wife swapping movie websites
  • sweet young pussy to lick
  • Natural big tits pics
  • fisting anal sex
  • young naked girl teens
  • thumbnails big black boobs
  • sucking fuck videos
  • mature women teaching young girls
  • adult couples nevada sex
  • videos of naked boys
  • lesbian twins have sex
  • blonde teen being fucked
  • cute jeans pussy
  • mother in law handjob videos
  • anna sex pictures marquette
  • tiny asian girl fuck video
  • stories of teenage girls spanked nude
  • berry halle naked video
  • orgasm sites xxx
  • Big boobs tight shirts
  • my wife seduced a young man
  • Lucy liu nude pics
  • bbw amateur sex
  • using fingers during oral sex
  • Big titts curvy asses
  • wife cummin vids
  • shemale tranny fist anal
  • nina young pictures
  • biggest butts in
  • japan av idl girls
  • cute teenage pussy
  • girls naked camera
  • naked sexy japanese girls
  • free ameteur couples
  • ghost sex with sleeping girl
  • free amateur emo girls
  • pussy n tits
  • free hot hardcore videos
  • naked simpson pussy
  • Naked topless girl
  • young models facial
  • free sophistied older lady sex pics
  • so so young girls sex tube
  • trapping cute girls into lesbian sex
  • young sexy swingers
  • drugs and sex videos
  • front seat oral sex
  • young ripe tits
  • Beautiful pussy videos
  • Chinese actress sex scene
  • asian girl gettting erotic massage video
  • Anal massage vids
  • large pussy small cock
  • young girl nude tits
  • Nude boobs of actress
  • Chubby teen dancing
  • Too much wanking
  • threesome vids
  • masuimi max sex videos
  • armpit girl suck
  • sexy bikini jessica simpson
  • pale redhead anal video
  • black bigg butt queens
  • young girls homemade sex video
  • wife bra pictures
  • fuck your girlfriend for cash
  • free young teen moves
  • teen anal sex fucking
  • muscular black man fucks wife
  • interracial sex small white girls
  • Pussy sewing video
  • beautiful girls young hq pic
  • school galleries
  • Megan fox nude movie
  • free comics video
  • download asian video
  • free videos of forced enemas
  • girls getting cum in ass
  • young lesbo girls kissing
  • illegal young girls photos
  • young girl erotica pussy pictures
  • Paris hilton big tits
  • young girl gives head
  • teen trampoline creampie sex
  • College girls having sex
  • girl tied up erotic tickle
  • young sex pics india
  • young girls doing deep throat
  • free teen fucks video
  • roleplay anal beads vids
  • petite blonde escorts
  • mature adult intertainment videos
  • hot white teen girl
  • perfect oral sex slutload
  • Photo big breast
  • free pussy sleeping videos
  • Indian teens gallery
  • sleeping black teens
  • sexe video amateur telecharger
  • lesbian bondage sex in shower
  • bloody free videos
  • lesbian gang attacks straight girls
  • kelly molnar sex pictures
  • thin skinny petite nude
  • black and white teenage girls
  • women bound and gagged videos
  • is clifton davis wife black
  • xxx creampie teens
  • naked girls doing cocaine
  • asian girl fuckking video