Category Archives: technology

YOW! Sydney 2019

Day 1

The Unicorn Project and the Five Ideals – Gene Kim

Gene brings lots of energy to his talk, and I’m a huge Gene Kim fan, so I had my books ready for him to sign. Gene paints a great vision from data showing the correlation between high performing companies and adoption of devops practices.

Evolving Chaos Engineering – Casey Rosenthal

Can you imagine what it would take to convince your boss to let you deliberately make things break in your production environment? Casey is the kind of guy who could make that argument. Casey is rapid-paced, will lots of gentle subtle humour, and some fun historical stories.

3 Insights From 4 Years At Spotify – Jason Yip

Everyone has had someone in their company want to ‘do’ the Spotify Model. What does it really look like? Jason gives a down-to-earth, human approach to solving organisation problems without the hype.

Multi-cloud, A Large Enterprise Perspective – Scott Shaw

Scott links together classic patterns we’ve seen in the software industry, that we’ve seen over and again over the last 30 years, and shows how they’re happening again with cloud vendors.

Interaction Protocols: It’s All About Good Manners – Martin Thompson

How is the Court of Versailles, Francis Bacon and the ITEF related? Come to this talk and find out. Martin rants wonderfully about the problems with the BSD sockets API and links it to a different way of thinking about problems.

Growing Your Personal Design Heuristics – Rebecca Wirfs-Brock

Can you distill the way other people think in concepts? Can you capture and repeat your own abstraction process? Rebecca brings decades of experience to the meta-problem of your own problem solving process.

The Reactive Revolution – Josh Long

It’s the end of the day and Josh’s energy hits you like a bolt of lightning. Every developer has seen Java. Josh does a demo that will knock your socks off. This was so much fun.

Scale, Micro-services and Flow – James Lewis

There is this inspiring theme that builds up through this day, starting with Gene Kim, about organisations living and dying as organisms. James kicks off with a great t-shirt reference, and links quickly to fractal networks. James is great fun.

Day 2

The Morning Talk Walkthrough

Aino and Michele – succeed in making the talks appealing, but also bringing in a dry-humour of their own. Make sure you ask Aino about the OODP talk she did in the past. (Hint – it’s not about Design Patterns). Michele is great to chat to during the day, and has put lots of thought into making the conference a great experience.

Rise of the Breaches – Troy Hunt

Troy Hunt wrote haveibeenpwned.com and has testified before the US Congress on Security matters. Security is both deeply serious, and knee-slappingly hilarious and Troy piles it on.

Quantum Computing and You – Matthew Keesan

Imagine the Computing World in the 1930s – before Von Neuman…

What are all the element of a good YOW talk? Energy. Jokes. A vision of a better future. Real world applications and shooting down myths. Feeling empowered and inspired to do it yourself. This talk ticks all the boxes. Probably the best non-keynote talk.

Food

In the 90s I imagined that IT people ate burgers and drank soft-drink. 20 years later we have a different kind of metabolism, and the conference caters to that. The Sydney food was artisanal and filling.

Automating Operations with Machine Learning – Matt Callanan

Pure, real-world, on the ground, battle-hardened solutions. Matt is neck-deep in this stuff and you can throw him any question.

Ready for Rust – Erik Dörnenburg

If you haven’t yet wrapped your head around the key concepts of Rust and its benefits – Erik packages it up neatly in 50 minutes, with some great demos. Erik brings the passion of one who enjoys programming, talks about industry applications, and drops some sick burns.

93 Easy Steps to DevOpsing Your Monolith – Cat Swetel

The room packed out for Cat with good reason. Cat delivers irony and sarcasm like few you’ve ever seen. Be ready for your sacred cows to be roasted, because Cat is going to tell you how it really is.

Designing Distributed Systems with TLA+ – Hillel Wayne

What does the complete state space of the interactions with your software system look like? Can you represent that as a 3D wireframe? What language would you use to describe that? People love computer languages, but this is not what you think.

How I learned to stop worrying and love Misery – Gil Tene

Gil Tene wrote the best JVM. What happened when he went to measure it with customers? Here are the principles he extracted from his experiences. A year ago I wouldn’t have needed this talk. This past 12 months I’ve been using this all year.

The 12 Aspects of Object Oriented Programming and Why Functional Programming Has A Better Answer. (According to Lawrence Krubner)

What is it?

These are a set of arguments suggesting that Object Oriented Programming has 12 attributes, the benefits and deficiencies of each, and that Functional Programming addresses each of those.

Where does this come from?

This is from a page on the internet for a site that has now been taken offline, but is preserved on The Wayback Machine

https://web.archive.org/web/20191004095642/http://www.smashcompany.com/technology/object-oriented-programming-is-an-expensive-disaster-which-must-end

What is the style of writing?

It is provocative and argumentative, seeking to make a point by exhaustive evidence and reasoning. The original title was “Object Oriented Programming is an Expensive Disaster that Must End”.


Why post this?

  • I studied Eiffel at University in the mid 90s, and read Bertrand Meyer’s books on Object Oriented Programming and was fascinated by the promise of Object Oriented Programming and what it offered.
  • In 2009 I became fascinated by the promise of functional programming, and how it offered an alternative to OOP which was the dominant paradigm at the time.
  • Most arguments in favour of Functional Programming start with “the future is multi-core, and functional programming has a better answer than OOP”. This author starts with: “Here are the 12 attributes of functional programming, why they’re good, why they’re bad, and why functional programming has a better answer.”
  • I don’t necessarily endorse the arguments of the author in their entirety, but am fascinated by the reasoning, and wanted to capture it and share it for further discussion.
  • I wanted to summarise the arguments in a table, because lots of people don’t appreciate reading large slabs of text.
  • I felt that the strength of the arguments was lost in the provocative style of the writer and felt the same arguments could be more convincingly presented.
  • Even Bertrand Meyer, one of the chief academic proponents of Object Oriented Programming, finds himself arguing against the excesses of OOP, making the arguments that match the benefits of functional programming

Is it pragmatic?

This is not answering the question “How do I make my program/lifestyle better?” It answers the question “What are the reasons you would consider Functional Programming, looking at the world from an Object Oriented Programming lens?”

Object Oriented Programming Characteristic OOP Benefit OOP Limitation Analogue from Functional Programming (or alternative to OOP)
1. Encapsulation Protection from unexpected changes to state. Solving the state management problem by hiding it. Use closures.

Design your program not to mutate state.

Design your program around transforming the state.
2. Polymorphism We want flexibility in the way we dispatch execution.   Flexible dispatch based on the signature of a method, and the signature is almost always evaluated in terms of the data-type of the parameters being handed into the method Multi-methods. [In Clojure. Typeclasses in Haskell are somewhat comparable.]
3. Inheritance Treating similar but slightly different things the same way externally.

Hierarchies of data types.
Inheritance mixes together several things, ‘typeness’ [sic], interface and implementation.

“Favour composition over inheritance” (Josh Bloch)
Define data-type hierarchy independently of functions and independently of any state.  
4. Abstraction Use abstraction to decompose complex systems into smaller components.

Denotes the essential characteristics of an object that distinguish it from all other kinds of object and thus provide crisply defined conceptual boundaries.   Dependency injection is an example of depending on abstractions.

Contract enforcement.

Data hiding.
Abstraction is not necessarily a subject within OOP.

Bertrand Myer admits that OOP abstractions lead to factories and builders, the complexity of which lead to Inversion of Control Containers, which lead to greater use of composition.

The need for design patterns implies a deficiency of abstraction in the language itself.
Solve your problem in terms of DSLs.  
Macro-systems can be a tool for abstraction.
5. Code Re-use Objects created for Object Oriented Programs can be reused in other programs.

Many types, one interface promotes code re-use.
Inheritance undermines code-reuse.   Code reuse killed off by the amount of ceremony required to get anything done. (Factory patterns, dependency injection, getters and setters, annotations, verbose exception handling, etc).

All the specificity of interfaces/classes kills your code re-use.

To achieve code re-use, you need to write a large number of small classes.

The rigidity of the Open-Closed principle tends to limit code re-use.
Protocols. [In Clojure.  Traits in Scala or Typeclasses in Haskell are somewhat comparable.]

Meta programming.

Closures.

Libraries.

Flexible functions that can work on many data-types.




6. Design Benefits OOP forces planning which leads to better designs with fewer flaws.

For large programs, OOP programs are easier to program than non-OOP ones.
‘Forced Planning’ is an appeal to Waterfall development method. Risky because you gain benefits at the end of a long process. Only at the end of the software delivery can your users give feedback about whether your software helps them.   Organisations choose adaptive not predictive development methods like Agile.
7. Software Maintenance Whilst more work went into designing an OOP program, less work is required to maintain it over time compared to a non-OOP program.   Maintainability depends on understandability, and OOP is a graph of mutating state that becomes less understandable as it grows.

OOP tends to be bloated, containing lots of boilerplate code.
Code written in a different style would be smaller and easier to think about.
8. Single responsibility principle A class should have only one reason to change [structure of class].

Objects can be clearly reasoned about because they have focus.
The reasons a class can change are driven by Conways Law (ie your program is shaped by the information flows in your organisation.)

Single responsibility principle, gives you more objects, and gives you the responsibility for setting up all the dependent objects for your object, implying the need for IOC containers.  
You should have functions that operate on a data structure.
9. Open/closed principle Enables low coupling, high cohesion. Tends towards having too many small classes.

Conflates three issues that should be separate:
1) establishing a data-type hierarchy
2) enforcing a contract
3) mutating state

Rigidity of this principle limits code re-use.
Use protocols. [In Clojure.  Traits in Scala or Typeclasses in Haskell are somewhat comparable.]
10. Interface segregation principle Many client-specific interfaces are better than one general-purpose interface.

No client should be forced to depend on methods it does not use.

Keeps a system decoupled, and so easier to refactor, change and deploy.

Useful ‘contract’ to describe the intent of the software.  
Data structures are spread out across multiple classes making the intent of the software harder to understand.


Have a well-defined data-structure that many functions can operate on.
11. Dependency inversion principle Helps to assemble components from different projects into a cohesive application.   Provides a common pattern for wiring components together in a decoupled way.

Solves the problem of instantiating the objects that an object depends upon.
Solves a problem that only exists in Object Oriented Programming. IOC is a workaround for OOP. Not required because the problem doesn’t exist outside OOP.
12. Static type checking Enable the efficient and automatic proof of assertions about programs.

Enable partial specification of your programs.
OOP languages lose the benefits of the most advanced type-system features available in functional languages. Use a functional language to get the benefits of the most advanced type-system features.  

There are exciting things happening with the Scala cats libraries and scalaz, inspired from Haskell that this author doesn’t address.

Reference

http://www.smashcompany.com/technology/object-oriented-programming-is-an-expensive-disaster-which-must-end

YOW Conference – Sydney 2018

This year was a celebration year – YOW has been running for 10 years! (I do remember being in an evening meetup 10 years ago in Sydney with Dave. He was testing the idea of having a conference and asking people if they’d come, and getting lots of nervous an uncertain looks.)

To me the consistent standard of talks is higher this year, than any of the other years of YOW I’ve seen over the last 10 years.

This is post sharing my experiences of the talks this year.

Thursday 29 Nov 2018

Kent Beck – 3X: Explore/Expand/Extract

@KentBeck

Kent is a hero for so many people. His methods were so influential. Along with the Agile Manifesto many people were given a vision that software development and delivery could and should be better.

The man in the flesh is none of that. He humbly explains the things he has learned, and even uses analogies from his career break.

This is a refreshing and inspiring talk. It is not what you expect.

Michael Nygard – Grinding the Monolith

@mtnygard


Always listen to the Clojure guys… they’re the out of the box thinkers! (This talk is not about Clojure).

Microservices were all the rage in tech talks a year ago, but the subject now can lead to an angry silence. Why?

They can be complex to build, complex to run and you only need them in some scenarios.

But Netflix did it!

What are the reasons and 8 successful and unsuccessful transition patterns? Michael delivers.

Kevlin Henney – 1968

@KevlinHenney


Kevlin is the kind of guy you’d want to go to the pub with, to hear some great stories. This talk feels like that. (Although I’m sure he’d love to have a beer and chat at the end of the day).

In the past I’d heard Kevlin roast SOLID design principles, so expectations were high.

This talk is a wonderful snapshot and encapsulation of Computer Science and Engineering history.

Sid Anand – Big Data, Fast Data @ PayPal

@r39132

Sid is a team leader who parachuted with a couple of guys into a part of Paypal that was bleeding and fixed it.

This is real. The legacy system diagrams are ugly.  The data contortions to make it work are eye-popping.

What I took away was that there are emerging some common patterns for real-time financial transaction customer experience at scale that aren’t simply ‘noSQL is the answer’.

Dave Cheney – Lessons learned building Kubernetes controllers

@davecheney

Everyone talks about Service Meshes. Some people have played with them. Dave commercialised it.

This is a technical story about and Aussie guy, part of a team who focused, and did the hard work of delivering. Good onya Dave.

Randy Shoup – Breaking Codes, Designing Jets, and Building Teams

@randyshoup

Have you ever wanted to visit Bletchley park and find out the environment that generated the computer? Have you ever wondered about the team culture at Lockheed’s Skunk Works that delivered the SR-71 Blackbird radar proof plane? What about the vibe at Xerox’s PARC that gave use the GUI, ethernet and WYSIWYG word processors?

Randy reckons you can have that in your team. Come and find out how.

Simon Raik-Allen – From the Caveman to the Spreadsheet and Beyond

@simonraikallen

Simon brings science fiction and recent IT nostalgia in a charming journey asking “What’s Next?” The William Gibson Neuromancer references are a strong theme.

The difference is, Simon delivers. I won’t spoil the surprise.

Anita Sengupta – The Future of High Speed Transportation

@doctor_astro


Anita is a rocket scientist and professor at the University of Southern California. When she speaks – the energy she projects makes you feel like you’re on one of those rockets.

Parachuting robots onto Mars? Done that. We’re onto something more interesting. [I won’t spoil the subject matter]

Anita finished by answering questions with flair, then when out of time, assuring everyone she’ll be grabbing a beer and that the conversations can continue.

Friday 30 Nov 2018

Jessica Kerr – The Origins of Opera and the Future of Programming

@jessitron

When you started your job, did you ever deal with an arrogant senior developer who didn’t facilitate an environment for learning?  Have you ever wondered why the renaissance was a big deal? Jessica links these two. This talk is breathtakingly original.

Jessica is a polished speaker with a lot of experience. I didn’t look at my watch for the hour of this talk.

This talk thematically links in with Randy Shoup’s talk on high performance teams and provides motivations and techniques do it.

Because the standard of this talk was so high, I dare to ponder how it could be better.  I couldn’t help feeling like Jessica stepped around a leadership model and a world view behind all this like it was the elephant in the room. Maybe it wasn’t appropriate for this audience.

Mikael Vidstedt – Java in a World of Containers

@MikaelVidstedt

Mikael is the hands on guy from Oracle, talking about new features in the JDK, showing you how to make your Java program run in a docker image more efficiently.

Now to be honest, making minimal docker images for java programs was a thing about 3 years ago.

But you never had the Oracle JVM guy telling you techniques and hacks you’d never heard of for shaving 10’s of megabytes off the remaining size of the docker images.

For this guy – the JDK not a fixed canonical tool to be used – he treats it casually as the output of a compilation. A thing to be sliced, diced and cooked up.

You may come out of this talk wanting to work for him.

Casey Rosenthal – Deprecating Simplicity

@caseyrosenthal


This guy lead the Chaos Engineering team at Netflix. He wrote the book on Chaos Engineering.  Yet he is calm, confident and well-reasoned.

Casey relates the problems with the 1986 Challenger Shuttle disaster to how large groups of people in a kitchen doing cooking,  deal with each other.

The best bit was at the end where he explained how to sell resilience engineering in your organisation by using a Winston Churchhill quote.

Brendan Gregg – Cloud Performance Root Cause Analysis at Netflix

@brendangregg


Brendan was introduced as ‘that guy you have have seen shouting at hard drives‘.  Brendan is an infrastructure guy and metrics geek – and has been given the world’s biggest scale infrastructure problems to analyse and solve at Netflix.

This talk is the full toolkit to analyse every performance problem that you could have on one of your servers. You get the list of techniques and the tools. (Even Brendan admits he can’t remember them all – and so shows you his diagrams he puts on his desk to remind him.)

Chris Richardson – Events and Commands: Developing Asynchronous Microservices

@crichardson


So you’ve got an existing microservice architecture and now one of your transaction boundaries crosses the line across two microservices? Oh you’d never do that. (Because it would be Terrifying!) But what if you have to?

Chris has fought that dragon and lived to tell the tale. In fact he has written a book about it. Come and get a very detailed look at several techniques and the tradeoffs associated with each).

Michelle Casbon – Kubeflow Explained: NLP Architectures on Kubernetes

@texasmichelle


Michelle confidently explained the sorts of problems you’d apply machine learning to. She comes across as polished and knowledgeable.

Then she rolls out this gem that connected with all parents:

“I was reviewing a pull request for this machine learning platform pipeline at home, and my five year old son was making paper aeroplanes and building bridges out of cushions…”

How does she manage this?

Then she goes back to showing you how to select particular machine learning model types for your problem and how you can see the correlation results in the pipeline.

Cat Swetel – The Metrics You Should Use (but probably don’t)

@catswetel


It’s a tough gig being last on the lineup but Cat delivers. She has a dry, rich sense of humour that tolerates no BS.

Other speakers in the past have touted fancy formulas or Deming quotes, as the way to bring manufacturing statistical discipline to agile workflows, but Cat gives the real thing. Not only has she read every Deming book – she even quotes a book from Deming’s friend (and then roasts him for it).

War stories galore, Cat points triumphantly to a lower dot in the middle of a multi-modal distribution (we just learned that term):

“See that!” see says triumphantly, “That was me!”

If you want to know what’s wrong with your estimation process, (with evidence including the authors and papers to back it up) Cat will smash it for you.

Conversations with Dave Thomas (conference organiser)

Dave moves from group to group during the lunches and tea breaks – and has such a rich background in technology. I find it such fun to draw upon it in conversation:

“Dave, you know how when Apple transitioned from the PowerPC to Intel, they ran a binary translator, Rosetta (in addition to dual binaries) to keep the the old apps running on their OS. Now that Intel x86 is running out of steam and ARM architectures are looking good – do you think binary translation will be the answer for backward compatibility or should they just go to virtual machines for emulation?

Dave didn’t miss a beat:

“Well in a former life I had worked in this. The solution that is the best of both worlds is really register machines. A company that has now folded that came close to that was Transmeta. ARM architectures do look good in terms of power consumption and CPU power. I expect that if they do go down that path virtual machines may solve that problem. But really they can push people to just recompile their apps for the new platform, and that would help them clean all the old apps off their platform that hold them back.”

I had the privilege of attending the speaker dinner – and without prompting several people stood up and said kind words about Dave. The one that struck me most was this:

“Dave has a heart for the developers in Australia. There is so much going on in technology, and he wanted to be able to bring that to Australian developers, so they wouldn’t be left out.

Summary

So your highly caffeinated boss asks you with a sharp-eyed expression on the day after you got back from the conference:

“So. What did you get out of it?”

You’ve got about 7 seconds to not bore him and make it sound valuable.

“It was great! They looked at high performance teams, Kubernetes and Docker, machine learning, resilient systems, better estimation and performance analysis techniques.”

Commit Level Coverage Reporting

What is it?

This enables coverage reporting showing lines not covered on each commit

Where is it?

https://github.com/juliangamble/commit-level-coverage-report

Is there an example project?

https://github.com/juliangamble/jacoco-example-project

Why you would use this

[Workplace with a million line codebase and 100s of developers working on it.]

Tech Lead: “We really want to get the coverage up on this codebase – more tests will give us more feedback about the code.”

Developer: “Well that’s lovely, but I really need to ship my feature – I’ll make sure my stuff is covered as best I can.”

Tech Lead: “Ok – how much coverage are you aiming for?”

Developer: “I’ll make sure the lines I touch have coverage.”

Tech Lead: “Ok – how will you measure that?” ….

Now obviously:

(a) this codebase is a candidate for modularisation and

(b) sonar does give you feedback on coverage at a commit level – but sonar doesn’t give you a way to know what the commit-level coverage is prior to pushing your changes. ie – if you’ve missed some coverage on the lines you’ve changed – there isn’t a way to know before pushing.

This change is trying to solve the problem of knowing “for the lines I have touched – what coverage have I got? What lines have I missed?”

How to setup and use in your project

Add the following into your pom.xml

<project>
    <build>
        <plugins>
            <plugin>
                <groupId>com.github.juliangamble</groupId>
                <artifactId>commit-level-coverage-report</artifactId>
                <version>1.0</version>
                <executions>
                    <execution>
                        <id>post-unit-test2</id>
                        <phase>test</phase>
                        <goals>
                            <goal>report-on-commit-coverage</goal>
                        </goals>
                        <configuration>
                            <!-- Optional - sets the path to the file which contains the execution data. -->
                            <!-- <dataFile>${project.build.directory}/coverage-reports/jacoco.exec</dataFile> -->
                            <!-- Sets the output directory for the code coverage report. -->
                            <outputDirectory>${project.reporting.outputDirectory}/</outputDirectory>
                            <daysBackToCheck>100</daysBackToCheck>
                        </configuration>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>
</project>

Then run this:

mvn clean test

Example Result

----
Commit: caaa32f8c8eff580b4760f60dbf021d6e03935c3 - Sun Jul 16 20:48:50 AEST 2017 - Julian Gamble - [dev] first drop

source file name: /com/machiavellian/MyApplication.java
Intersection of line changes with coverage (lines we care about): 4
covered lines: 3
Coverage for commit: 75%
Lines not covered: 1
src/main/java/com/machiavellian/MyApplication.java:11         return firstArg - secondArg;

----
Commit: 57bfb7956b501afca5fc101753d6e284d448f4ce - Sun Jul 16 20:53:59 AEST 2017 - Julian Gamble - [dev] adding new method

Intersection of line changes with coverage (lines we care about): 1
covered lines: 1
Coverage for commit: 100%
Lines not covered: 0
----
Commit: acf35680891d1f853a226035981c1ac18d9f50a2 - Sun Jul 16 20:55:58 AEST 2017 - Julian Gamble - [dev] adding coverage

Intersection of line changes with coverage (lines we care about): 0
covered lines: 0
Coverage for commit: 0%
Lines not covered: 0
----
Commit: 9c42a1ab1e8db57e31e17837120da6dc257dbb72 - Sun Jul 16 20:56:23 AEST 2017 - Julian Gamble - [dev] adding divide method

Intersection of line changes with coverage (lines we care about): 1
covered lines: 0
Coverage for commit: 0%
Lines not covered: 1
src/main/java/com/machiavellian/MyApplication.java:19         return firstArg / secondArg;

----

Result File Location

target/site/coverage-per-commit.txt