One of the main reasons why it’s difficult to troubleshoot problems in complex environments is lack of documentation. This post is long overdue because it was inspired a while ago by Dan Fink’s post (again). Be sure to read it first.

From time to time, I see situations when one person leaving the company leads to a disaster that puts in danger the whole existence of multimillion businesses. Why? Because there is nobody else who knows how system works and how all its moving parts are connected. How does that happen? No knowledge sharing.

The most universal method of knowledge transfer is documenting stuff and I would argue that, most of the times, one hour of documenting brings to a business more than two hours developing new functionality in PL/SQL.

Actually, there might not even exist such know-it-all-wizard. Complex systems are not created in a matter of days or even months. Often, extreme complexity is the result of years or even decades of evolution. All those workarounds, bug fixes, re-designs, pressing deadlines — those friends of entropy win time from documentation tasks.

How many times it happened to you:
- Why are there forty-two underscore parameters in every new database you create?
- Because it’s our standard.
- Can I look at it?
- {proudly} Yes. It’s in every init.ora file we have!

Or another variation:

- Can I look at it?
- {proudly} Yes. Here is 42 pages document how to create new database:

Performance boosters:
_db_aging_touch_time=…
_db_aging_stay_count=…
_db_writer_chunk_writes=…
_discrete_transactions_enabled=…

Bugs:
_kgl_latch_count=…
_kgl_bucket_count=…
_lm_rcfg_timeout=…

and etc…

- When was this document started?
- {proudly} Oh! Forty-two years ago!
- When was it last time updated?
- Eh… I’m actually quite new here…

What are the reasons behind missing or inadequate documentation?
Just to name a few:
- It’s boring.
- We have no time for it now and it’s not a priority (it never will be) — we still missing couple reports.
- I’ll do it tomorrow.
- Project manager is not paid by number of pages of documentation produced (not that this is a good idea either).
- I’m the only one who knows this system — they must be idiots to get rid of me.
I’m sure readers can name plenty of others and please feel free to do so.

Sometimes you can avoid complexity and make systems simple by design but in real life it’s more often when you need to deal with complexity. Paraphrasing Dan — documentation will not make systems simpler but will reduce perceived complexity.

If you’ve got to support and/or develop a complex system — how do you make sure you dedicate enough time to documentation? What are the strategies to keep it up to date? How to explain the importance of documenting to the management? If you have anything to say — the comment box is at the bottom of this screen — use it. If you think that you have enough to say for the blog post — just let me know.

Find more about  , , ,

7 Responses to “Avoiding Guesswork in Complex Environments”  

  1. 1 Christian Antognini

    Amen!

  2. 2 Frits Hoogland

    Good points Alex! Exactly the last situation I was partly involved with.

    In my experience, setting up a successful system, either complex or simple, requires a systematic approach from the start. And, which cannot be said enough, simplicity!

    A database needs to be adjusted as little as possible. Sourcecode needs to be version controlled, Even in very small simple environments. Production environments need software releases, explicitly tagged from the version control. Not a few files to be fiddled with in order to make it work again.

    Happy easter!

  3. 3 Narendra

    Alex,

    Nice thoughts.
    I agree completely.
    Your last reason is very important. People, many times, do not share knowledge, due to insecurity. Sounds funny, but it is a fact. They (probably) get “married” to the company and whenever they are acknowledged as S.P.O.C. (Single Point-Of-Contact), it soothes their ego.
    Such people will do everything they can, to sabotage any plans to prepare / update documents.

  4. 4 Aleksey Tsalolikhin

    Documentation is very important and under-used in today’s IT shops.

    To help remedy that, here’s my recent post on system administrator site documentation with docbook, and a few “intro to docbook” links that were helpful to me.

  5. 5 Alex Gorbachev

    Good point. Thanks Aleksey.
    The biggest difficulty I found is to motivate engineers to write docs AND keep them up to date.

  6. 6 Graham Oakes

    Excellent points Alex.

    I would also add that writing the documentation is only useful if you ensure it’s properly distributed or can be easily found. Finding the correct location for the documentation and ensuring that it’s organised in such a way to make retrieval easy is not as simple as it could be in a large company(at least that’s what I’m observing where I work atm).

  7. 7 Alex Gorbachev

    Thanks Graham. I might be biased or just used to Oracle Database documentation style but after comparing Oracle database documentation with documentation for the products that Oracle acquired — I can only attest to your point. The ability to navigate quickly and effectively within a large documentation volumes is a must.

    But it’s not only the documents creators who should invest efforts, readers must invest enough to be familiar with organization and knowledge domain. Even when search functionality works great, you would still need to know what to search for.

    Thanks again for the feedback.

Leave a Reply


  • realy hot realy young topless babes
  • bible lessons for teenage girls
  • young blonde masterbating in public bathroom
  • hardcore modle for in australia
  • xxx adult free hardcore fucking
  • teen chirstain movies
  • picture egories
  • husband slave wife mistress pics
  • girl flashing big boobs
  • xhamster big pussy lips mature
  • petite latina mandingo
  • Adrianne bailon nude pictures
  • young japan idol movies
  • young mothers pussy
  • schizophrenia wife missing branagh movie
  • pictures exposed breasts
  • shemale sex movies galleries
  • Pamela anderson nude images
  • old guy teen girl sex
  • movies of sexy black teen feet
  • amateur teeny facialized
  • pregnant black chicks fucking
  • Brooke shields nude movie
  • ava vincent anal sex
  • Watch best sex scenes
  • sexy young girls thumbs
  • Ass licking photos
  • naked videos of francine prieto
  • Minka tit fuck
  • young teenagers flashing boobs tube
  • spreading amateur gallery
  • pics naked milfs
  • can a pussy get strep throat
  • young girls suck big dicks
  • kinky lesbian sex
  • mature young group sex
  • India sex gril
  • girls naked facial
  • Indian hot sex scandal
  • forced painful anal video
  • partylite girls candle
  • two young to fuck
  • old asian man fuck teen
  • small waist with big tits movies
  • watching couple fuck
  • womens dresses petite plus size pattern
  • wife swapping sex iubes
  • newleywed and sex video
  • slavegirl wife humiliation
  • hien tits pussy
  • chicks pics babe
  • fake david henrie naked pictures
  • i wanna taste my ass video
  • barbie party ethnic
  • sweet young pussy video
  • amateur wife cumming on my dick
  • hot lesbians eat pussy redtube
  • hot teen naked pictures
  • black that ass up
  • free swingers homemade videos
  • teen nudist naked nude art young
  • free movie women fuck men
  • nina hartley vintage interracial tubes
  • wife breast feeding pictures
  • family guy free cartoon movies
  • pics of youngest pregneant girl
  • Nude emo girl pics
  • girls first black
  • colorado interracial sex
  • free sex pics and vids
  • Angelina jolie fake nude pics
  • hand rail in pussy
  • xxx young teens
  • movie girl with giant thumb
  • xxx young trannys
  • naked girls of warcraft
  • beautiful girl titty fuck movie
  • you girl cumming
  • Prego sex
  • Naked muscle girl
  • free old men sucking nipples movies
  • Indian best sex scene
  • huge black blonde group
  • India boobs sex
  • naked girls of facebook
  • white girl pussy fucking
  • next door niki pussy
  • Tits lesbian pics
  • annabella angel pussy
  • best japan xxx
  • you black on white
  • voyeur beach sex videos
  • asian categories
  • pussy videos for
  • Kathy griffin naked pics
  • kimono girls hardcore dvd
  • girls bikini japan
  • insane amateur sex acts
  • teen having sex video
  • black on asian sex
  • indan wife sex desi
  • free hot young sex videos
  • older wife weing night fuck
  • black girls eating black girls pussy
  • petite flatchested girls
  • kiss ass lesbian babes
  • ass spread galleries
  • Sex scenes tara reid
  • group sex parties
  • Mariah carey sex photo
  • pregnant star pics
  • Hot girl black
  • anal disney
  • sexy young girls on cma
  • looking up little girls dress pictures
  • tied naked public sexy
  • mature mom home video
  • Nude actresses videos
  • Christian anal sex
  • married hairy wife orgasm creampie
  • Lesbians lick feet
  • bdsm teen hardcore
  • free anal young
  • young thai girl fuck movies
  • free pics of young nude celebrities
  • naked flat chested teen girls
  • asian personals sex
  • amateur mature german classic wife boss
  • bbw teen young fat
  • free online videos of insest
  • video femme mature fist fucking
  • beach pictures teen girls
  • blackwomen take it in ass
  • nude ebony teen pussy
  • sexy girl strip dance
  • pretty little naked girls
  • thai erotic massage videos
  • interracial sex and cuckoldry
  • tits dicks free
  • teen girl bibkini model pics
  • naked pictures of your wife
  • sex and sex teens
  • nancy erminia pictures naked nude
  • meghan heffern naked pics
  • Indian live cam sex
  • Sex in the city girls nude
  • hrine tate bouncing boobs video
  • anal sex causes prolapse
  • japanese xxx teen
  • shemale femdom anal spanking
  • Mutual oral sex
  • white sexy tits
  • naked pics of eva mendez
  • girl pics thongs feet
  • painful pussy fisting
  • little black girl lost
  • free erotic spanking videos
  • free asian women movies
  • streaming webcam
  • tits size shape gallery
  • naughty sex kits young
  • sniff my dirty girl ass
  • Girl drinks glass of cum
  • hot blonde has sex with stuent
  • pictures of star justine joli
  • rescue me sex scene videos
  • Nude island video
  • black phat ass girls
  • Hot girl dancing naked
  • naked videos of pretty girls
  • frist anal free
  • wifeswapping spanking novel club anal
  • Girl on girl naked
  • youth and beauty models pictureview
  • black dick gangbang
  • video girfriend naked
  • Lady death nude
  • pregnant pussy during pregnancy
  • free white gangbang vids
  • high definition facial cumshot videos
  • sexy babes girls wifes
  • video nude teen girl
  • black chicks cum
  • free sex pics and movie
  • fuck my wife blindfolde spread eagle
  • retro big ass boobs gallery free
  • watching wife fuck
  • very young k pics
  • Lesbian sex tits
  • public girls flashing
  • college girls and pics
  • teen girl sex viedio
  • pussy eating lessons
  • teen boys sex stories
  • young boy with bogs sex
  • japanese blowjob in cars
  • teen strips webcam
  • Hot and sexy emo girls
  • syracuse young sex
  • longer flash sex videos
  • asian school girl cartoon
  • dy little girl virgin
  • video of rescue me sex scene
  • Older hairy pussies
  • teens first fuck movie
  • gossip girl blonde actress sings
  • upost sex videos
  • sex video asian
  • pauly shore naked pictures
  • Mallu reshma nude sex
  • black girls fucks big black cocks
  • asian teen trannie
  • free russian oral cumshots
  • sexy midwest girl
  • hardcore three way fuck movie
  • free chubby nude girls movies
  • teen interracial videos free
  • orgasim young girls tube
  • free amateur flashplayer
  • amaxing blonde you
  • girls gone wild picss
  • transsexual group sex
  • topless vegas shows videos
  • black cum in wife tube
  • girls first time eating pussy
  • amateur sex orgasm video
  • dancers girls movies
  • brothers that suck dick
  • hard lesbian sex pictures
  • Home made indian sex video
  • crazy granny pussy
  • very young lesbian port
  • asian sex vedio
  • girl gallrey
  • dailymotion blonde college girl sexy
  • different ways lesbians have sex
  • naked stickam girld
  • busty teen rachel video
  • oral amber blowjob movies
  • Perverse sex videos
  • group sex kitchen
  • asian war
  • petite gagging cumshot
  • white guys gang bang black guy
  • large wife sex party videos
  • slender ebony hooker fuck vids
  • teen double vag triple vag sex
  • Fucking a passed out girl
  • best blowjob galleries
  • Huge bbw breasts
  • transsexual melanie videos shemale yum
  • ass worship free vids
  • crochet pattern petite irish rose
  • xxx serials cracked passwords
  • Hot oily sex
  • best ebony gloryhole streaming
  • crystal ray throat gaggers videos
  • perfect sexy ass
  • mature big tits fuck milf sex
  • young mscular girls
  • mature long vids
  • amateur wife deep throat
  • sexy girl jumping
  • calender girls helen mirren nipples
  • girl kneeling down to suck cock
  • dasani teen hitchicker videos
  • shemale anal veggie
  • brazilian pussy videos
  • ms black nude beauty
  • karen martin girl next door pictures
  • sexy lesbian pussy sex
  • big black butts hanging out
  • homemade video wife loud orgasm
  • latex blowjob movies
  • redhead mature galleries
  • snowtrails bikini pics
  • pussy pumped camel toe
  • inez teen model free pics
  • my black fuck
  • petite juene fille
  • amateur anal post
  • streaming sex masturbate girls
  • ass wife galleries
  • amputee sex pics
  • black rat snake pictures
  • wife swap home movies
  • bushy black girls
  • free video hardcore busty babe
  • tiny little virgin teen girl
  • teen naked myspace
  • black fuck milf wiesbaden mainz georgina
  • pussy pumps in my area
  • glamour babies girls glamour
  • xxx asian panties
  • Next door nikki naked videos
  • petite rolo chain
  • black breeding rack videos
  • pussy big tits viedos beach