XP Redux

This is about software development, specifically the project management thereof, executed with a pattern of small iterations. This is a brief re-hash of Extreme Programming, based on my limited experience with such practices to-date.

A colleague asked how I'd do it, so here's the general idea.

Outline

• A project is defined as something that has many small iterations of start-to-finish, instead of one big start and one big finish.
• Depending on the size of the project, iterations can be hours long (hackathon), or they can be months long (multi-MM $ projects).
• For the sake of parsimony, we'll assume that it's preferable for iterations to be regular (i.e. they are all of the same length). You may experiment with irregular iterations at your leisure.

Specifications

In software, the conventional divide is between the two following levels of abstraction.

• Functional Specification
  
  This is the higher level of abstraction. The project targets are described conceptually. More detail is better, and ideally there should be no loose ends. These could be implemented as follows.
  1. brainstorming: random thoughts
  2. analysis: remove contradictions
  3. analysis: map out major dependencies; necessary and/or sufficient conditions for success
  4. analysis: map out every single dependency
• User Interface (UI) Specification
  
  This is the lower level of abstraction. The entire surface of the software-to-be-developed is specified in space and time - if any part of the software is perceptible to the user, this is the time to decide on how exactly that will happen. This could be implemented as follows.
  1. doodling: rough sketches
  2. analysis: remove contradictions
  3. analysis: map out dependencies; necessary and/or sufficient conditions for success
  4. final art: major states of affairs
  5. final art: every possible state of affairs

The steps in blue are ideally completed at this stage, but are often procrastinated to step (Iterating 2.) in the schema below. Following the feasibility study, the depth of which is an art to be mastered, there should be enough information with which to divide the project's wish list into iterations.

Before Iterating

Prior to iterating, moderately detailed quantitative estimate should be conducted. This is often referred to as a feasibility study. It involves one or more quick and dirty surveys of the project's specifications.

Iterating

Each iteration could look like this, for example:

1. A review of the entire project's specifications at both levels of abstraction, progress (if any) in previous iterations, and a revisiting of whether each outstanding item should go into the current iteration, a future iteration in this project, or into a "future project"  (when a project has been broken into discrete groups of iterations, each group may be referred to as a "phase").
   
2. Final specification of work to be developed in this iteration (from the final blue steps of the feasibility study, otherwise you now have to execute the blue steps; if you ignore the blue steps completely, you risk feature creep and meandering timelines; in web development, for example, ideally you've already got CSS and HTML).
3. Development.
   
   • Automated testing - only if things pass, they move to the next step. [CI]
   • User-acceptance testing (UAT) - only if things pass, they move to the next step. [CI]
   • Product-owner signs off only on specifications that are considered complete.

Continuous Integration

If you practice continuous integration (CI), that happens throughout stage (Iterating 3.). If you don't, then you must allot time for it elsewhere, and therein lies more uncertainty, and therein lies the reason that CI is a preferred practice.

Continuous Deployment

If you practice continuous deployment (CD), that happens in each iteration after stage (Iterating 3.). If you don't, then you must allot time for it elsewhere, and therein lies further risk, and therein lies the reason that CD is a preferred practice. This pattern is possible only when the use-cases of the software are tolerant of failure, and when the cost of deployment is trivial - you don't want to try this if you're planning the first mission to Mars, for example.

A Light Friday Night

So, it's been a quiet week. I've actually had the space to "go out," for coffee on a Friday night. Partly, this degree of freedom was purchased through a huge rise in rental opex. Before that, I went shopping and bought a laundry basket. Tomorrow, I must do laundry, while draughting some slides to pitch for a job in machine learning. At some point over the weekend, I must spend some time improving my ability to deploys some tools we will use on the current project. A modicum of responsibility, many opportunities to contribute to other people's well-being, a luxurious degree of freedom (by global standards), and no sulky bitches to entertain... for now, blessings counted. A good day to die. Still nailin the KPIs. So off then, to find some dinner. Rinse, and repeat.

Machine Training

Not sure what some of these AI researchers are thinking.

Training a machine to recognise cups by showing it pictures of 50,000 cups is like teaching a kid to recognise cups without ever showing them what a cup does. You need a volumetric projection, stupid...

---

There's really nothing wrong with neural nets. or iterative self-pruning hidden layers of any kind... but in this case, we're talking about 3D objects being analysed in 2D without a 3D data-structure.

I know it's expensive to do things in 3D, but this is swasacbkards.

ASPD?

A recent reminder:

I'm quite aware that smiling makes people receptive to information - I also generally don't care for people who don't make themselves receptive to information.

So, as I'm not naturally smiley on most trivial matters (business in general), I save energy by not smiling, and prefer to let nature take its course with people who don't pay attention to what I'm saying.

That describes my anti-social preferences quite well - so maybe I am an introvert after all. But I so maintain that it's mainly for the want of adept conversants.

Keep Moving

Next job starts in two days, but the office opens tomorrow, so I'll be there.

Air quality rubbish. Nose running. Hard to study.

Which means I have only sufficient mental powress to gibber with acquaintances over coffee, while sending drinks to strange women at the bar.

In a minimal way, that is productivity :)

Ansible Training

What.

---

Chef Training

What.

---

Puppet Training

What.

---

Vagrant Training

What.

Related

Packer - configure once, virtualise anywhere (currently: Amazon EC2, DigitalOcean, VirtualBox, and VMware)

Other HashiCorp products

---

Deep Learning? Bah Humbug. But I Guess it could Pay the Bills

Hi X,

I saw your ad on Y for this role. The role interests me, but i will not be available for a full-time commitment for the next six months, as I'm about to start a contract of that duration. My CV is attached, but it is a strange one, so you'll have to tell me if there's any chance you'd like to work with me in the future.

Here's what's not in my CV

I pursued "gradschool," in software development, finance, and mathematics, autodidactically, from 2008 to 2013:

Programming

You'll find that I have a fair amount of experience in software development (full-stack) but not a lot in C++ and Python - I've dabbled in those, and never had to use those before, but in the long term that's not a problem as skills will improve with practice.

The lowest-level language I've spent at least a few months working in... is Haskell; the largest project I worked on in Haskell was a web-development framework comprising web-server, MVC architecture, cookies, sessions, and connections to a MongoDB database.

Before that, I studied Erlang/OTP in order to make use of its applications in distributed computing - it turned out to be quite fun to work in for binary/text processing due to a language construct called IO-lists. I did study the Princetonian WordNet project, and practiced Erlang by scripting an extraction of all the data from it (https://github.com/jerng/wordnet-studies).

Limitations of past work in mathematics/statistics

My appreciation for robotics goes as far as this: a good robot will be able to do all the following things better than humans have in the past: invent new hypotheses, critique existing science, rewrite physics and mathematics from arbitrarily defined foundations (these are after all, linguistic representations of facts - facts are fixed, but an infinite set of representations is possible, and standard theory is just one of them).

Accordingly, the majority of my (very limited) work in mathematics post-high-school has been in the area of foundations - how it all works, abstract algebra, category theory, etc. (Coincidentally related to my study of the Haskell programming language.)

My bachelor's degree is in Philosophy (2005).

I was primarily interested in the information architecture of university syllabi because it seemed that the academic organisation itself wasn't very MECE.

I ended up working on the quantification of human experience, and concluded that 80s-00s AI efforts seemed to get the data structures wrong... high-profile AI work of this era puts too much emphasis on finding relationships between morphological constructs, and not enough on performing operations on representations of sensual data. So I think in the long run, "deep learning" will only work when we're representing human thought as models of sensation, and not mere networks of words. I'll keep working on it in my spare time, if I don't develop a career in this field.

Over to you. I'm happy to meet in person or on VOIP for a chat.

Lies

Studying for the next job, while on vacation. I told someone it's because I don't have a girlfriend. That was a half-lie. I'd actually prefer a girlfriend who appreciates work as much as I do.

(So this is all scraped from a thread on FB.)

What do you mean, what's a girlfriend to me?

Spending time together is fine. Outside of physical connectivity, I believe it would be productive to discuss work, how to do it better, and how to actually do some together.

I wasn't designed to be much of a consumer, unfortunately. I tend to behave like a weapon. My latest movie crush is pretty much Faora lol.

Why are we even talking about workaholism?

This isn't "workaholism" - workaholism is defined as an addiction to whatever X is considered to be work, which results detrimental side-effects to the individual's health / social life / well-being etc.

This is Sparta. We're fitter, happier, more productive, probably better in the sack (if you take the long-term average :P), and definitely avoidant of mating outside of similarly performant circles... LOL

I'm only HALF joking here...

... what's not to joke about: it seems absolutely likely that we'll die off because we took obscenely stupid risks out of boredom, or from trolling some juggernaut civilisation and dying in the process with a big broad smile on our faces.

The Anti-sale

While I'm quite happy to spend this decade prioritising money over education, I'm probably going to retain the modus operandi of discouraging potential customers/clients/employers from dreaming about profits without clear rationale.

Let's just call it a long-standing positioning decision.

10-year prioritisation of stupid stuff over smart stuff. 15% in, and I feel that my thirties are going to evaporate before I know it.

Docker Training

Run it on...

any Linux (kernal version >= 3.8), on...

• Bare Metal

• Vagrant

  • DifferencesProvisioning Docker on Vagrant
  • Vagrant and Docker
  • Differences: Docker vs. Vagrant

Repositories / Imaging

• Docker Registries
• Dogestry - an attempt to improve efficiency / simplicity.

Deployment / Production

• Misconceptions - TL;DR: don't pursue premature optimisation
• Writing the logic - list of tools and approaches
• ambassador pattern - containers that serve as gateways for inter-host container communication
• OpenShift - a point of view

Bugs

Inconsistent persistence of REPO:TAG metadata

p
There appears to be strange behaviour, only partially fixed, in the persistence of the REPO:TAG meta-data of images, across save and load operations. This.

Volatile configuration warning

Related to this. Currently (docker-1.1.2), if, while in a container, you change something that happens to be in the domain of the config file (e.g. environmental variables), exit the container, and commit the changes to an image(a)... then those changes are lost. Some possible work-arounds:

1. The "right" way (not always possible):
   
   • after committing an image(a),
   • docker run [specify config here, e.g. -e HOME=/root/ -e X=123 -e Z=abc] the image(a),
   • then commit it to an image(b).
   • The second image will retain the required configurations.
2. If you can't (1.) then the emergency method is:
   
   • after committing an image(a),
   • save the image(a) to a .tar,
   • un-tar(a) it,
   • find the most recent json file,
   • manually tweak the config files,
   • re-tar(b) it,
   • then load back the image(b). (good lord...)
3. (off topic, but relevant) using .bashrc properly
   
   • if you're using an image based on one of the public Ubuntu images at Docker Hub,
   • then when you execute docker run OPTIONS IMAGE bash,
   • include -e HOME=/asyoulikeit/...
   • ...and bash will load /asyoulikeit/.bashrc
   • (this is just another way to set up an approach to (1.)

---

'To-do's

1. Figure out how AUFS and Docker work, so that backups can be done more effectively, instead of using save for every image.
2. Figure out port forwarding from host->container, and container->host.

Passion? Nah

I don't love solving problems. I just do it because it's not boring. And it pays ok.

[Edit: originally, "solving problems," was written as, "programming."]

For context, I suppose I find that:

- sales pays great, but is usually boring;

- physical activity is highly entertaining, but it pays less, on average.

Maybe I should sign up with a football team.