|
|
Harpreet Singh |
I will do a series a blog as I get my head wrapped around Kubernetes.
A recent proclamation in the technology trends is that Kubernetes has won the container wars. It implies that Kubernetes is the new operating system for the cloud.
One of the key misconceptions is that k8s is an orchestration engine for docker containers. This is how I thought about it. That is true but it is much more.
Container orchestration is a key capability of the platform but there is much more. I think of k8s as the container orchestrator and the ecosystem services around it. Here are the key services that a company needs to think about as they bring in k8s:
So if you are bringing k8s in-house or evaluating a cloud solution, you need to think about the above categories of service and make sure that your provider is providing an acceptable service (for you) on each of the categories.
My interview on the trends for the next 5 years in CRN, Germany.
TL:DR - Curiosity, learning, distributed teams, kubernetes, devops and machine learning are going to be the key drivers in the next 5 years.
You know her; you have seen him in action; and too often you have been one — a superhero who flew down, laid yourself on the broken tracks and let the train pass over your shoulders and saved the day.
Have you been one? I know I have.
There is an escalation from a customer that has to be solved and a colleague has worked round the clock to bring it into the release schedule and will soon the solution will make it’s way to the customer.
Absolutely the right thing to do!
So what’s wrong with this approach?
The answer to my follow up question “so what moved out to make way for this escalation?” and the answer was “nothing, we pulled it in”.
Why is this a problem?
Occasionally becoming a superhero is a much desired attribute in a team member as it shows initiative, dependability in a team member. Done too often it shows organisational issues.
A superhero anti-pattern is an opportunity to unmask the following issues
The team is under-staffed
Too often, the team has a charter way bigger than they can handle and people need to be superheroes all the time. This over a period of time is going to impact the team credibility, morale and general well-being of the team members.
Not enough processes
Process is a maligned word but a good process in place is the world of difference between a teams ability to scale and respond to changes.
Too many escalations by too many stakeholders
Every person in the organisation has a direct access to any of the team members. “After all open communications is our thing!” The problem is that the team ends up responding and clarifying things to every person within the org and that sets the team up for failure. Recently, a colleague introduced me to the role called “the disturbed” which nominates a person on the team that takes the disturbances for a short period of time and keeps everyone focussed on their tasks. In this case, we are nominating one super-hero proactively.
The team leader and the team is not learning
There aren’t enough retrospectives in play that help the team learn and improve.
The individual loves the superhero mode
Almost everyone enjoys responding to the urgent versus doing the important because too often the important thing is really hard to do and there is always enough time. Urgent on the other hand makes people look good — who doesn’t want to look good?
The individual is gaming the system and climbing the ladder
Raise enough red-flags, come in, save the day and look good — win kudos. This motivation is insidious and eats at the culture of the organisation and the morale of the team. Rooting this out is hard and can be done encouraging members to continually focus on the important and not the urgent. When urgency itself is not rewarded the behaviour will be greatly reduced.
The superhero anti-pattern rears its head (often) in organisations that are undergoing rapid scaling. The superheroes themselves will not have enough bandwidth to recognise the sub-optimal behaviour and its ramification on the organisations. It is an opportunity for leaders to step back and unmask underlying organisation issues.
So what about my colleague — turns out that all it took was a conversation to point out that lets not put undue stress on the system and indicate what downstream deliverables get impacted and reflect that in the plan. Once done it provided a breathing space to the team to confidently execute on their deliverables.
I was fascinated when Facebook launched the feature where it put a box around a human head (and a bit creeped out when it started suggesting the name of the human next to the box). I always wondered how they did it and filed it under machine-learning magic-ery. Now, I know how they do it so let me peel back the curtain.
As product managers, product roadmapping and planning is one of the most rewarding activities that we do in our jobs. Determining the most valuable problems to solve and finding the shortest path to bringing them to the market is the engine that drives business growth.
A softmax function squashes its inputs to be between 0 and 1 and also normalises them so that the sum is equal to 1.
Let’s take a problem where we have to classify between three classes.
Softmax is used to classify for greater than 2 classes for less than 3, use a sigmoid function. So, we need to tell if an animal is a duck, a dog or a cat. Our Neural Network linear function looks a picture of an animal and comes back with say a score 2 (duck), 1(dog) and -1(cat). The next step is to convert the scores into a probabilities whether the picture is a duck, dog or a cat.
This is where Softmax comes in. The softmax function takes independent scores and comes back with probabilities such that the sum of these probabilities is equal to 1. Thus, we can classify a picture against these classes relative to each other.
The formula is
$$ P(duck) = \frac{e^d}{e^d + e^o + e^c}$$
where d is duck, o is dOg and c is cat in the formula. Our end result will look something like:
P(duck) = 0.70
P(dog) = 0.26
P(cat) = 0.03
Thus, we have taken a matrix of scores and converted them into probability distributions $$\begin{bmatrix} 2 & 1 & -1 \\ \end{bmatrix} $$ Apply softmax to produce $$\begin{bmatrix} 0.7 \\ 0.26 \\ 0.03 \\ \end{bmatrix} $$ Thus, to summarise, a softmax function squashes its inputs called logits to be between 0 and 1 and also normalises them so that the sum is equal to 1.
Let’s say that there are a bunch of events and a bunch of probabilities, cross entropy predicts how likely are those events likely to happen based on the probabilities.
A high cross entropy means that an event is unlikely to happen while a low event indicates that the event will happen.
In a neural network (NN), an error function is used to classify data into appropriate classifications. So in well performing NN the goal is to minimise the output of the error function. Another way to look at this minimization is that we should be increasing the probabilities that these points are classified right. Thus, we are aiming to get the probabilities of the events to get higher.
Let’s take an example - where we have 3 doors and if we open the door, we are likely to either find a duck, dog or a cat behind each door. These events are independent to each other, thus all 3 doors can have a dog behind them.
Let’s say the most likely probability is to find the following behind each doors:
Door 1 = P(duck) = 0.70
Door 2 = P(dog) = 0.80
Door 3 = P(cat) = 0.97
Thus, the likely hood of finding a duck, a dog and a cat is $$ 0.7*0.8*0.97 = 0.6 $$ Since multiplication of various events with numbers less that 1 could be very small, we typically use the logarithm function to add them up and that is the formula for Cross entropy $$Cross entropy = -log(0.7) - log(0.8) - log(0.9)$$ $$Cross entropy = -(-0.35566) - (-0.2231) - (-0.1053)$$ $$Cross entropy = 0.685$$ This low cross entropy number indicates that the likelihood that these events (of a duck, dog, cat ) showing up behind the doors is high. To implement this in tensorflow, you'd do the following: ```python softmax_data = [0.7, 0.2, 0.1] one_hot_data = [1.0, 0.0, 0.0] softmax = tf.placeholder(tf.float32) one_hot = tf.placeholder(tf.float32) cross_entropy = -tf.reduce_sum(tf.multiply(one_hot, tf.log(softmax))) ```
I believe the audience for this blog in the world is a grand total of less than a couple of dozen and fewer who would understand the desire to chase Turiya - I frankly don’t understand it myself :-). Thus, I am documenting this as a reminder to myself for a few years down the line. If you want to know what an advanced state of meditation looks like - read on.
Twenty years ago - pre-internet era, I read a book called “Living with Himalayan Masters” by Swami Rama, a Guru of “Advaita Sri Vidya”. This book had fantastical stories of Yogic masters performing incredible feats. I bought it hook-line and sinker, perhaps, because I had just lost my younger sister and I was looking for a grander explanation to life than “it just the way it is”. Never found a teacher, because Swami Rama had passed away in 1996.
Nevertheless, this was the first time I came across “Turiya - the 4th” and have been on its hunt ever since.
Advaita (non-duality, formless) philosophy posits the universe as one single consciousness or universal consciousness and each human as a manifestation of the universal consciousness. Though, as with anything hinduism, oxymorons abound just as “hai bhi aur nahin bhi” or “it exists and doesn’t exist at the same time”;”Advaita” is achieved through “Dvaita” or duality or in this case the “Goddess and the God” - take your pick - literally - take a pick of what you resonate towards (no God - consciousness, God, Godess). The goal is to experience universal consciousness. The easiest way to think about consciousness in the Advaita world is it is the “force” that is behind everything (Star Wars anyone!). Turiya is the best way to experience it and meditation takes you there.
Thus, meditation (Dvaita) –> leads to –> meditation (Advaita) –> leads to –> Turiya –> makes you experience –> universal consciousness.
So last week, I attended a week of intense practice of “Advaita Sri Vidya” practice with Avdhoot Shivanand and after years of off-and-on practice I finally consistently hit Turiya and hopefully can hit it without being in the presence of the master. Side note: Avdhoot Shivanand is perhaps the only living master who teaches Advaita Sri Vidya today.
Typically, we think of three states of consciousness:
The Upanishads describe Turiya as the fourth state of consciousness that is behind the three states. It is beyond the typical dreaming, sleeping or waking states, on in which, you are completely relaxed, calm, composed and active at the same time. The rate of flow of thoughts flowing through your mind is close to zero. You are aware and in the Here and Now. Timelessness is a fundamental quality of this state.
This state is the one that Yogis achieve to maintain all the time because of supposedly innumerable benefits to mental and physical health. That said, the prime attachment to this state is because you are aware and constantly in touch with universal consciousness.
I liken it to the “flow” state which athletes, musicians get into when they do deep practice. They, unfortunately can maintain this only for a few minutes but a Yogi in meditation can be in it for hours. As an ex-programmer, I experienced this state when in deep coding where I lost track of time.
The experience is on so many different levels and hard to explain - there is no existing frame of reference to compare it with so bear with me.
First, I did lose track of time. Everyday, we went into meditation for what seemed like a 15-20 minute session and then when we opened our eyes - it was about 2-3 hours into the meditation. The longest session was about 3.5 hours. Note: I have a back that gives me problems after about 15 minutes in one posture and here we were on one of the more uncomfortable chairs that you see in conference rooms and I was rocking it.
I have been in Buddhist practices, where they take your watch away and make you close your eyes for practice to experience time and unfortunately you are aware of every excruciating minute of that practice. But not Turiya - you walk out of a Turiya session and go “what the hell! I could’ve stayed there for an hour more - lets get back in”!
As I mentioned, I could sit through long sessions without much discomfort. The desire to move wasn’t there at all. The breathing is very fine and very “deep”. Deep breathing has real benefits on your health and I guess I accrued some :-). Utter and complete relaxation is perhaps a better description of the physical state. This relaxation is a better quality than one where you get up after long sleep. I wasn’t drowsy; I was awake; Here and Now.
Awake; Here and Now.
Typically, the mind is racing at a few dozen thoughts a minute. In Turiya, thoughts completely disappeared or were coming at a very slow rate.
Why is this important?
Advaita says that thoughts have either a happiness or unhappiness quality to them. You are either in the past or in the future and rarely here either reflecting on an unhappy or a happy past or future. In Turiya, the thoughts that come have neither quality, their quality is observational (my words). You can dispassionately observe them and insights flow in. It is almost like connecting a diagnostic tool to your computer - you just know what needs to happen to fix that problematic pattern you have been carrying around. Often, I was “observing myself” as a third person which is a deeply reflective state and resulted in interesting experiential insights.
That said, absence of thought is more often experienced than presence and analysis of thoughts.
The experience of universal consciousness was truly other wordly. I don’t quite have words to describe it. The gates to universal consciousness start from body consciousness and the meditation session started by making me very aware of my body consciousness (hard to describe) and at some point a metaphorical gate opened and I entered a deeper state and I realised that I was experiencing universal consciousness. How do you know it is universal consciousness - you just know - deep in your bones. Weird!
The closest description is when in the move “The Matrix”, Neo realises that he is the One. He realises that the matrix is running underneath him and the agents and they are the same. That description gets you close enough.
It felt like I was plugged into a battery and getting my charge in. Typically, I don’t realise that I am connected but now I was completely aware that I am being charged and could “see” the source. I use “see” as “see it in the third eye”.
Finally, the “I” disappeared even if it was for a short time and I became part of the universal expression where me and the chair and then neighbour and her chair were all one. Just one part of this fabric called the universe existing together Here and Now. This state is called “Tat tvam asi” or “Thou are that” or “being one with the force but better ;-)”.
Seems like that the entire experience was perhaps less than 5% of what you aspire to.
More practice ahead :-).
Building a scrollable component in react native is fairly easy but comes with a gotcha.
Simple enough. I had a View component that looked like the following:
<View>
{ this.renderAlbums() }
</View>
This was replaced with
<ScrollView>
{ this.renderAlbums() }
</ScrollView>
The gotcha was that I had to change the top level application code to bring in a style with a flex of 1. I don’t quite know why!
const App = () => (
<View style= >
<Header headerText={'Albums'} />
<AlbumList />
</View>
);
Unlike web apps React Native doesn’t have universal styling that is specified via a css. React solves it by asking developers to marry the styling within the component file. As a developer, I found this very convenient to work with - I knew exactly what styling I had to bring into my component. That said, I would expect this to be a challenge working with a designer requiring a constant back and forth. Here is an example of a component that I put together.
import React from 'react';
import { View } from 'react-native';
const Card = (props) => {
return (
<View style={styles.containerStyle}>
{props.children}
</View>
);
};
const styles = {
containerStyle: {
borderWidth: 1,
borderRadius: 2,
borderColor: '#ddd',
borderBottomWidth: 0,
shadowColor: '#000',
shadowOffset: { width: 0, height: 2 },
shadowOpacity: 0.1,
shadowRadius: 2,
elevation: 1,
marginLeft: 5,
marginRight: 5,
marginTop: 10
}
};
export default Card;
Recently, I came across a pictorial that showed all the products in the DevOps market and the sheer number was mind boggling. As the head of products at CloudBees, I am often asked about recipes to build standout products in such markets and I will share my thoughts here. I will speak in the context of the DevOps space but the lessons should apply elsewhere.
A challenge in this space is that tools have been built by the tool users themselves or in other terms “experts have built for experts”. DevOps has been a ground up movement, tools are born because someone scratched their own itch.
That is wonderful - the market space is validated! However, the early movement has become a democracy now - everyone is doing it. This implies that novices expect to come in and be productive but the tools aren’t suited for them. This is an opportunity to differentiate!
A corollary of the above problem is that these tools have been around for dozen or more years and just like the roots of a tree have spread across the entire organisation and integrate with every conceivable tool in there. This is good news for your product from an adoption perspective but it literally means that you have to change the tires of a running car if you want to rethink your product. Me-too implementations can rethink the experience but usually end up as implementing toy use cases because it will take them a generation to catch-up and build for enterprise tools. This is an advantage for existing tools and an opportunity to differentiate!
Standing out from other product requires building products that are clearly differentiated. There are a number of product levers that can be played around with to build this differentiation. Levers like ease of use, targeted use cases, better branding, lower cost. I will focus on the product because if the product isn’t right - fixing other things is like putting lipstick on the pig.
An organising product principle is the guiding light-house that aligns the entire organisation. For example, at CloudBees, ours is “Build a lovable product experience”. We constantly ask this of ourselves and use it to make tradeoffs in hard situations. This principle is highly energising and motivating - who doesn’t want to build something kick-ass and the principle sparks of very interesting and deep discussions in the company
What’s your organising principle? If you cannot articulate it - start here first.
You always have to remember that there is human on the other side who is going to use the product. Too often, enterprise products solve for all possible personas with the intention to make everyone happy. This results in user experience that isn’t targeted enough and consequently lovable enough or in other words clear-as-mud :-). Too often, I see that product teams forget about the persona and it shows in the product. At CloudBees, everyone from the exec team, marketing, sales and ofcourse product management knows the names of these personas, their motivations and even their looks. We not only know them, but target product requirements for that persona. We trade-off decisions based on the persona. It isn’t uncommon to hear statements like “Ada would never use this, this feature is for Simon and we aren’t focussed on Simon in this release”.
So, know the persona and make it as real as possible for a differentiated product.
Who is your user? Where does she sit in the organisation that you are selling to?
Who is your buyer? What do they do after work?
What are the right problems?
Once you know the persona, understanding the key problems is crucial. Building a good body of problems (at CloudBees we call it problem statements) is a matter of interviewing the target persona - rinse and repeat. The lean startup movement is exclusively focused on finding the right problems and I encourage you to read it.
How often does your user have the problem that you are solving for?
Simplify - don’t solve for corner cases. Solve the 80% really well and provide an escape hatch for the remaining 20%. Most devops product want to solve 100% of use cases 100% of the time. This leads to really complex products that hurt the novices coming in. Think how can you reduce the surface area so that people are not taking customers into a dark alley. Reduce the decision points that customers make. More decisions equals paralysis and leads to a frustrating experience.
As a side note: In the DevOps world, products cannot ignore the last 20% because this is where advanced use cases are solved. The key is to provide extensibility points so that customers can scratch their own itch. Fundamentally, this implies architecting the product so that it is extensible and implies factoring this architecture choice early in the vision of the product. Make a product extensible isn’t something that can be tacked on. If you are evaluating a product in the DevOps world, ensure that it is extensible.
Simplification becomes easy once you have the persona and the right problem statements because you can ask the right questions. Are you adding more value by allowing the persona to do more or are you simplifying a persona’s life by reducing the steps in the process that the persona goes through every day.
I strongly advocate for a market validation and research phase that happens prior to execution. This phase is usually for small, medium and large initiatives and not for a bug fix for example.
Build mockups, prototypes and iterate. Iterate with design, iterate with engineering, iterate with support, iterate with sales and most importantly iterate with customers. This before any substantial piece of code is written. By the time a new feature set lands at engineering, it has to be vetted and certifiably lovable. Having an engaged customer advisory boards that have a good representation of your target personas is a compelling tool in a company’s tool set. Note: I am not advocating waterfall or agile - this is somewhere in the middle more towards agile.
A lovable product isn’t going to be delivered if the right team isn’t in the picture. The team should be a triumvirate of product, design and engineering. The roles should be clearly defined for the triumvirate. Invest in design, product management and engineering meaningfully. Too often, this part of the organisation is running at capacity keeping the fires down and then executives wonder why good products aren’t being delivered.
The product professional owns the persona and the problems. She knows what needs to be solved for and has an idea how that should be solved but should not go about designing the solution. A product manager’s job should then ensure that engineering and design “get” the problem, closely work on building the right solution and validating the right solution. A product manager mocking something on paper and throwing it to engineering is perhaps the worst thing that you can do to build differentiated products.
Design takes the problem and builds the solution and its user experience. The role of the product manager is to provide the right customer context so that the designer can design the right solution. Design should be empowered to stand up to engineering, product management and should own the user experience.
Engineering comes in and keeps designers and product managers real. They help them understand the costs of the choices being made and then build them. By the time, the product feature is in execution, engineering should know the customer, the problem and design intimately.
Finally, provide time and space for these individuals to ruminate and think. Interesting solutions happen when you are bored and have taken your focus away from the daily fires. At CloudBees, product managers, designers have a day called “Product Leader Friday” to work on projects that expand the understanding of a persona for themselves or educate the knowledge of internal teams or evangelise something they care about. I am writing this article on my product leader friday!
Standing out in a crowded market does not happen magically. It requires deep product thinking, organisational frameworks and executive buy-in and support to make it happen. It then requires relentless work guided by organising principles that have been laid out earlier. That said, if I summarise things, the nub really is about three key questions:
Who am I solving the problem for? What problem should I solve for her? What’s the best way to solve her problem?
Simple enough!
Having a stressful day at work? If so, use the Aana Paana meditation as your go-to technique to find your calm and centre and deal with your day. It takes about 10 minutes to centre you.
The Lotus heart meditation calms your nerves, addresses insomnia and helps adjust to jet-lag. Let’s how you can bring this technique in to your life.
(Photo credit: Tom Woodward)
In the last blog, I gave a very quick lay of the land of meditation. In this blog, I will get you started quickly. So quickly and so easily that you will wonder why you haven’t done meditation in the past!
Recently, a couple of my closest friends have been recommended to start meditation from their physician as a means for improving their general well-being. My friends asked me a few questions about it and the line of questioning leads me to believe that it is highly likely that they are going to struggle. A few other friends have also asked me on how best to get started so I figured I will use this opportunity to write about beginning meditation.
Let’s look at a problem: I am a dog lover and I subscribe to Instagram where I get a stream of images of toasters, birds, cats and dogs. While these images are exciting (who doesn’t like toasters :-)), what I want to do, is automatically save images of dogs to an account on the cloud. The question is, what kind of program do I need to write that tags all dog images to feed my cloud database of awesome-dog-images.
Getting to what customers want, not what they think they want, is key to the success of a product manager and consequently to the success of the product itself.
This blog captures an intuitive understanding of concepts such as Regression analysis required for deeplearning.
Recently I wrote 2 blogs (tell me more, be an intern) talking about tips to conduct an effective customer/prospect interview for product managers. This blog wraps up the tail end of that series.
One of the most effective power questions is “What else is on your mind”.
There are some things that completely blow your mind. Applying artist specific styles to images is one of them. Completely sci-fiction - although the Prisma filter has made this accessible.
This blog sets up the core software requirements for the Udacity deeplearning course to get you started quickly.
Finding the right problem, its details, from customer interviews are similar to finding an address in India.
A problem while looking for an address in India is that neighbourhoods are congested and the roads are not labeled. One uses landmarks to reach to the right area, find a shopkeeper to guide you to the next smaller neighbourhood landmark - rinse, repeat - recursion :-).
Last week, I visited a CloudBees customer with Kohsuke Kawaguchi (Jenkins’ founder) to interview them. The goal was to understand their key challenges and concerns.
Today, continuous delivery (CD) is en-vogue. As companies adopt CD, they are faced with a choice on the tools to establish their delivery pipelines. Jenkins is often the obvious choice and best of all it is likely used by the engineering division already. This blog looks at various data points to paint a picture of the pervasiveness of Jenkins for CD.
Delivering software continuously, in other words “keeping software in an always shippable state” is a challenge for companies. Recently released Jenkins 2.0, is focussed on solving this problem through Pipelines. This blog gives a quick overview of key benefits of Jenkins Pipelines.
I have spent the last 3 years of my life helping Jenkins claim its place upfront and central in the Continuous Delivery market. The journey first started by providing first class support for CD pipelines through Jenkins Pipelines and has culminated in the first major release of Jenkins in 10 years. I am incredibly privileged to have helped Kohsuke Kawaguchi in this journey. The following is a blog post that I put out on cloudbees.com for the release.
Well. Doing my personal blog again…last I did it was at Sun Microsystems maybe 10 years ago.