# restart and run all or it didn't happen
my golden rule for notebooks is <q><b>Restart and run all</b> or <i>it didn't happen</i></q>. <abbr title="Restart and run all">RARA</abbr>||<abbr title="Get The Fart Out">GTFO</abbr> for short.
the <abbr>RARA</abbr> criteria is satisfied when:
1. all of the cells are executed. conversely, there are not any unexecuted cells.
2. all of the cells prompt numbers increase monotonically. out-of-order prompts means out-of-order execution and the possibility of hidden state.
![a text based figure displaying RARA GTFO RARA GTFO laid out as four square on a checkerboard](attachment:7fab95cb-30d3-4b4b-80cd-e32d1bc4b9b0.png)
## benefits of <abbr title="{{RARA.strip("_")}}">RARA</abbr>||<abbr title="Get The Fart Out">GTFO</abbr>acomputationalnotebookisa[literateprogram]thatcompositesnarrativeandcode;itsimultaneouslyhasthequalitiesof__literature__anda__program__.literateprogramsintroduceacomputationalqualitiestodocuments,andthecomputeimpliedtheauthoringsystemhad[state].[acommoncomplaintisthatliteratenotebookprogramscontainhiddenstates][complaint].hiddenstateishappenswhencodecellsareexecutedout-of-order,ornotexecutedatall.executingcodecellsensuresthatcodecanbereadbyatleastonemachine;thecodeinputworkedatsomepoint.orderedexecutionisanobjectivemeasureofnon-hiddenstate[^tamper].intheliterature,[ExplorationandExplanationinComputationalNotebooks],alargescaleanalysisofnotebooks,theydistinguishbetweenlinearnotebooksandnon-linearnotebooks.non-linearnotebooksaremostlikelytohave[hiddenstate](https://ploomber.io/blog/nbs-myths/#hidden-state).atscale,{{RARA.lower()}}isadistinguishingqualityofcomputationalnotebooks.>[Themostprevalentexpressionofliteratecomputingrightnowisthecomputationalnotebook.][landscape][fernandoperez]-co-founderof[ProjectJupyter]-acknowledgedthatnotebooksareliterateprogramswithsomethingextra.hedescribesnotebookauthoringas[literatecomputing]wherelivecomputationiswovenintothedocument.thenotebookdocumentisstateful,statefulnessisaqualityofthisnewmedia.the{{RARA.lower()}}criteriaverifiesthatcodeinputsworkandthereisnothiddenstate.[^tamper]:itispossiblethatinputsweretamperedwith,butletusassumethebestintentions.[literatecomputing]:https://web.archive.org/web/20220510083647/http://blog.fperez.org/2013/04/literate-computing-and-computational.html[landscape]:https://www.ppig.org/files/2019-PPIG-30th-fog.pdf[complaint]:https://towardsdatascience.com/the-case-against-the-jupyter-notebook-d4da17e97243[storyinnotebook]:https://marybethkery.com/projects/Verdant/Kery-The-Story-in-the-Notebook-Exploratory-Data-Science-using-a-Literate-Programming-Tool.pdf[literateprogram]:https://en.wikipedia.org/wiki/Literate_programming[fernandoperez]:https://en.wikipedia.org/wiki/Fernando_P%C3%A9rez_(software_developer)[ProjectJupyter]:https://jupyter.org/[ExplorationandExplanationinComputationalNotebooks]:https://adamrule.com/files/papers/chi_2018_computational_notebooks_camera_ready.pdf[state]:https://en.wikipedia.org/wiki/State_(computer_science)
## {{RARA}} criteria implementationdefrestart_and_run_all(file,code_cell_count=0):`restart_and_run_all`isviolatedwhenweiteratethroughthethenotebookcellsfori,cellinenumerate((notebook:=read_file(file)).cells):ifcell["cell_type"]=="code":todiscoveracodecell:code_cell_count+=1try:*thathasnotbeenexecutedbecausethepromptisnotanintegercurrent_count=int(cell.execution_count)exceptValueError:raiseUnexecutedCell(F"cell {i} not executed")ifcode_cell_count!=current_count:*hascellpromptsthatdonotalignwithcurrentcellcountraiseOutOfOrderExecution(F"cell {i} is executed out of order")
### {{RARA}} exceptionsweassigntwoformal`BaseException`sfor`restart_and_run_all`:`OutOfOrderExection`and`UnexecutedCell`#### unexecuted cellsunexecutedcellsarerepresentationsonlybythepreformattedtextinput.theylackanoutputsorpromptnumber.withoutthesefeaturesthecodestandsonitsown.itrevertsbacktopseudocode.classUnexecutedCell(BaseException):`UnexecutedCell`sraiseswhenthecodecellexecutioncountsisanon-integer.#### out of order executiontherearesometoolsthataredesignedtobereactiveorexecutedoutoforder.theRARAcriteriondoesnotdismissthesecasesratheritcouldbeastartingstate.wemaysuspecthiddenstateclassOutOfOrderExection:`OutOfOrderExection`raiseswhencodecellexecutioncountsarenotmonotonicallyincreasing.defread_file(file):ifnotisinstance(file,str):file=file.__file__withopen(file)asf:returnnbformat.v4.reads(f.read())### extra preservativesnotebooksareswissarmyknivesdocuments;theyhaveapplicationastests,modules,scriptsanddocuments.the{{RARA.lower()}}criteriacombinedwiththe[differentcellexecutionstates][states]:`INTERACTIVE`,`SCRIPT`,or`MODULE`.assertrestart_and_run_all("2023-01-09-notebooks-states.ipynb")isNone,\
errorsareraisedwhenfailing,andidon't want to demostrate failure herewhenweassertthatapreviousnotebookwill{{RARA.lower()}}webecomeslightlymoreconfidentaboutthecodeinthatdocument.[state]:2023-01-09-notebooks-states.ipynb
## impacts of {{RARA}}-ability### {{RARA}} and reusabilityrestartandrunalliscriticaltoreusingpythonnotebooksas[modules,testsorscripts].thereareafewwaysof[testingnotebooks],andnoneofthesemethodssuceedwithoutnotebooksthatrestartandrunall.{{RARA.lower()}}asapracticewillencourageyoutowritenotebooksyou'll be able to return to.[modules,testsorscripts]:2023-01-09-notebooks-states.ipynb[testingnotebooks]:https://nbviewer.org/gist/tonyfast/e7dd7ff3d808d57b77e9765626a73a91
### <abbr>RARA</abbr> conflicts#### too many ideasitiseasytoput[toomanyunitsofthoughtintoanotebook](#Restart-and-run-all-roots). thisstrugglecanbestrikingwhenstickingtotherestartandrunallprinciple.whenwewritedocumentsaboutsingleunitsofthoughtwearetreadinginformaltestingterritory.#### compute intensive operationsapushagainst{{RARA.lower()}}ishavinglargedatasetsorothercomputeintensiveoperations.weapplythe<abbrtitle="cache rules everything around me">CREAM</abbr>principleandcacheourcomputeintensivetasks.inthesesituations,authorswillbenefitdataengineeringthereworktoavoidcostlyrepetitivetasks;you'll save your future self time.
## 📣 write notebooks that endure{{rara}}||<abbr>GTFO</abbr>helpsustrustour's and other'snotebooksmore.wecanbeslightlymoreconfidentinfutureusesofthatwork.{{rara}}.i'm cheering for y'all.iseey'all beating the snot out of every line of code.let's make these efforts endure. remember to tell you friends <q>{{RARA.strip("_")}} or <abbr>GTFO</abbr>.</q>
## devils 😈 share### {{RARA}} rootspaconathanwasthefirstpersoniheardthisconceptfrom.at[JupyterDay2016],atthetime,hewasusingnotebooksasmediumforimmersiveprofessionalpublications.theseoriolenotebookswereeducationaltoolsthatembeddedtheteacherandtheruntimetogether.thereweresomebeautifulvideosproducedwithsomebignameslike:...[paco]presentedwhathisteamlearnedwhen[producingteachingvideoswithnotebooks][oriole].>1.focusonaconcise"unit of thought">1.investthetimeandeditorialefforttocreateagoodintroduction>1.keepyournarrativesimpleandreasonablylinear>1."chunk"boththetextandthecodeintounderstandableparts>1.alternatebetweentext,code,output,furtherlinks,etc.>1.leveragemarkdownbyprovidinginterestinglinksforbackground,deep-dive,etc.>1.codecellsshouldnotbelong,<10lines>1.codecellsmustshowthatthey've run, producing at least some output>1.loaddatafromthecontainer,notthenetwork>1.__clearalloutputthen"Run All"--oritdidn't happen__>1.videonarratives:there's text, and there'ssubtext...>1.pauseaftereach"beat"--smile,breathe,allowpeopletofollowyouandthere,clearasday,inthetenthbullet,wefind__clearalloutputthen"Run All"--oritdidn't happen__.ilovethatthispatternwasdiscoveredwhenproducingvideocontent.itmeansthat{{RARA}}iscriticaltocommunicatingourideaswithnotebooksasthesubstrate.[oriole]:https://github.com/ceteri/oriole_jupyterday_atl/blob/master/oriole_talk.ipynb[paco]:https://github.com/ceteri[JupyterDay2016]:https://jupyterday-atlanta-2016.github.io/#paconathan