Pointers to prepare for that system design interview

1. Change data capture systems
• Brooklyn built by Linkedin
• Wormwhole by Facebook
2. Zookeeper
• Zookeeper overview
• Zookeeper recipes
3. Apache Kafka
4. Apache Samza
5. Apache Spark
6. YARN
7. Kubernetes

Links

• Grokking the System Design Interview
• https://github.com/donnemartin/system-design-primer
• https://github.com/checkcheckzz/system-design-interview
• https://github.com/shashank88/system_design
• https://gist.github.com/vasanthk/485d1c25737e8e72759f
• http://www.puncsky.com/blog/2016/02/14/crack-the-system-design-interview/
• https://www.palantir.com/2011/10/how-to-rock-a-systems-design-interview/
• http://blog.gainlo.co/index.php/2017/04/13/system-design-interviews-part-ii-complete-guide-google-interview-preparation/
• https://github.com/yangshun/tech-interview-handbook/tree/master/experimental/design

What are the good attributes of a new Senior Manager ?

Are you a new Senior Manager/Manager of Managers ? Did you only manage Individual Contributors(ICs) and senior ICs before and are you going to manage Managers for the first time ? Or is your team growing very fast and you are about to manage more managers.

There are a lot of articles to help out folks converting from IC to management. However, I have found few articles on being a good Senior Manager managing other managers. The goal of this article is help you bootstrap and avoid some common pitfalls. Hopefully you will also appreciate the difference between managing ICs and Managers. This is one of the most common pitfalls of new Senior Managers and the part that they find hardest during the transition.

A common question that engineers ask is what does my skip do ? This article should be a good read for engineers to understand your skip better.

There are two broader dimensions (2Ps) : Project and People that we will talk about.

Project 

1. As a Senior Manager, you need to manage a manager by setting expectations based on business metrics and outcomes. You have to give flexibility and freedom to the manager about how to achieve these goals. If you start thinking too much about the "how" then you will end up micromanaging the manager. You have to detach from the project perspective and only think from outcome perspective. The manager should have the independence to create an appropriate roadmap, work with PM, determine team composition and structure, setup collaborations to hit the goals. 
2. On the other side, while managing ICs as a Senior Manager, you may be more attached and involved from the project perspective. This is how the nature of the job may look different while managing other Managers vs other ICs. You will probably have a few senior ICs reporting to you directly and hopefully through them you are personally attached to only the most important projects in your team. This will help you scale as the team grows.
3. On the first point, I mentioned that you need to detach from the "how". That is both true and not true. It is more nuanced. You can regulate the "how" and this is your way of setting technical direction/culture of the team. Here are some examples of regulations to set : 

the operational efficiency of the team should improve quarter over quarter(reliability SLAs, oncall load, system outages, weekly post mortems) 

the engineering quality should meet minimum bar(test coverage above x%, design docs and processes, deployment schedules, ab testing guidelines) 

data quality standards 

These regulations will also determine the engineering culture in your organization and hopefully this is also what you promote and reward in performance reviews. This is what you need to be in sync with your new Manager report because this will be a key part of the performance conversations. The regulations don't always need to be top down. These can be and should be bottoms up from the Manager and the ICs. This brings us to the next dimension : People

People

As a Senior Manager, you set/establish the culture of your team. People are a big part of the culture and who you hire is key to maintain and grow that culture. While you draw major components from your company and organization(director's) culture, you should also work together with your manager reports and senior ICs and customize it as you see fit. From here on, you have to give the Manager the freedom to hire/retain/manage the people they think are fit for the team and for achieving the team goals while thriving and growing the team culture. You should be available to help them navigate edge cases whenever needed, for eg: a sell call to help close that rockstar candidate.

There are a few ways you can regulate the people side and make sure things are on check without micromanaging the manager :

• do regular skip level 1:1s to keep a tab on the pulse of the team. conducting good skip level 1:1s is an art and takes years of experience. that itself could warrant a complete post
• every company has some kind of manager survey that they conduct. make sure that the top concerns expressed in those surveys are regularly being addressed by the Manager. This will be your way of growing the Manager reporting to you and this will ensure that the team trusts you and feels that their feedback is actionable

This post is geared towards new Senior Managers. I have deliberately left out other aspects when a Senior Managers/Director has several Managers reporting to him. These managers may be collaborating internally and externally to hit some of their goals. Those are more advanced topics and create a different set of dynamics. I will save that for another post.

If you are a Senior Manager/Manager of Managers and have read this far, I would like to thank you for your time. I would love to hear your thoughts about this article, things you wish someone had told you when you started out and anything else that could benefit the community.

For engineers reading this article, hope this sheds some light into the life of your skip. Do you know now what your skip does ? Would love to learn more. Please leave your stories in comments.

Areas of investment by Big Tech

Mobile Hardware : Apple, Google
Mobile OS and App store : Apple, Google
Voice : Apple, Google, Amazon
Home(Camera, speaker, other devices) : Apple, Google, Amazon
Social : Facebook
Communications : Apple, Facebook, Google
Video : Google, Amazon, Facebook
Search : Google, Microsoft
Ads : Google, Facebook, Amazon
Cloud : Apple, Google, Microsoft
Computer Hardware : Microsoft, Apple
Computer OS : Apple, Microsoft
Productivity Suite : Microsoft, Google
e-Commerce : Amazon, Facebook

How big tech makes money - 2019 edition ?

How Big Tech Makes Money

This is the followup to the earlier version of the post which describes 2018 revenues and profits https://siliconvalleystories.blogspot.com/2019/08/how-big-tech-makes-money.html

Apple (Revenue in 2019: $267.7 billion, $55 billion)

• Q4, 2019 revenue - $91.8 billion, net income - $22.2 billion
• Q4 revenue breakdown by category
• iPhone - $55.9 billion
• Mac - $7.56 billion
• iPad - $5.9 billion
• Wearables - $10 billion
• Services(Apple TV, AppleCare, ApplePay) - $12.7 billion (up from $7.3 billion in Q4 2018)
• Active installed user base 1.5 billion
• Commentary : You can see that apple revenue is pretty much constant from 2018, however the stock is up because apple has consistently reduced the outstanding shares by buybacks which increases earnings pre shares. Also Services and Wearables revenue is blowout and increasing as Apple pivots itself into a services play from a phone maker. 

Amazon (Revenue in 2019: $280.5 billion)

• Q4 2019 GMV - $87.4 billion, operating income - $3.8 billion, net income - $3.2 billion
• Investing in Amazon provides significant international exposure. NA sales : $53.6 billion, international sales : $23.8 billion
• AWS 2019 revenue : 35 billion
• AWS Q4 2019 revenue : $9.9 billion Q4 2019 profits : $2.6 billion
• Commentary : AWS makes up most of the profits for amazon and accounts for 11% of the revenue. The retail business is a thin margin business and AWS remains the cash cow. Amazon and Apple have similar 2019 revenues, but Apple is way more profitable as a business. Amazon's AWS made in Q4 as much as Google's cloud made in whole of 2019 demonstrating its lead in the cloud. Amazon also declared 150 million prime subscribers which gives them a $16.5 billion recurring revenue bundle. 

Microsoft (Revenue in 2019: $126 billion, $39 billion)

• Q4 2019 revenue : $36.9 billion growing at 14%, net income : $11.6 billion growing at 38%
• Productivity and Business Processes was $11.8 billion and increased 17%(includes Linkedin, Office 365 commercial, Office 365 consumer, Dynamic 365)
• Intelligent Cloud was $11.9 billion and increased 27% (Server products and Azure)
• Server products and cloud services revenue increased 30% (up 32% in constant currency) driven by Azure revenue growth of 62%
• Personal Computing was $13.2 billion and increased 2%(Windows, XBox, Search, Surface)
• Commentary : Azure continues to be be bright spot for Microsoft and that continues to drive the revenue growth. Microsoft has Office 365 which is its recurring revenue bundle like Prime and it is using its sales team to drive Azure growth. 

Google (Revenue in 2019 : $160 billion, Net income : $34 billion)

• Q4 2019 revenue : $46 billion, Net income : $8.9 billion
• Google search - 27 billion
• Youtube - 4.7 billion
• Google network member properties - 6 billion
• Google Cloud - 2.6 billion
• Google Other (Includes Youtube subscription) - 5.2 billion
• Commentary : Cloud reached around a 10 billion annual run rate. Youtube generated 15 billion ads revenue. Google is making more revenue than microsoft but is a thinner margin business. However, Google's revenue is growing at 20% pace for the last few years, whereas Microsoft is around 15% growth rate. Youtube ads grew at 36% YoY. Google cloud is growing revenue at 50% YoY, however, the base in much smaller than AWS. To put things into perspective, AWS made in Q4 what Google Cloud made in 2019. 

Facebook(2019 Revenue : $70.6 billion, Net Income : $18.4 billion)

• Facebook makes most of its money from ads (98.5%)
• Q4 revenue - 21 billion, Net income - 7.3 billion
• Facebook DAU - 1.66 billion
• Facebook family of products MAU - 2.89 billion
• Commentary : Facebook family MAU is 2.89 billion compared to Apple's 1.5 billion devices. 

Overall Commentary

• Apple and Amazon have the highest revenue. Apple's total revenue is not changing much YoY, Amazon is still growing revenue at >20% clip YoY
• Apple makes the largest amount of net income every year. Apple returns largest cash back to the shareholders
• Google total revenue is now greater than Microsoft with Search alone generating close to 100 billion in 2019. Microsoft is a higher margin business than Google
• Facebook continues to boast highest percentage growth in revenue numbers at 27%

thoughts about self-worth, entitlement, resilience, endurance

Performance reviews, 360 degree feedback, fast feedback is common theme in today's corporate america. Employers today care less about well being of employees. Workplaces are stressful and challenging. Businesses are way more dynamic than ever before. This trickles down to less planning and more adaptivity, be it in your roadmap, your career growth, your next promotion. There is always uncertainty.

In this age it is ever important to understand the difference between self-worth, entitlement, resilience and endurance. The difference between the four can be trick and the balance is key to good emotional health.

Self worth : Firstly, your year end performance is not a measure of your self worth. The true measurement of self worth is not how a person feels about their positive experiences but rather how they feel about their negative experiences. You have probably heard about the self esteem movement : feelings of self-esteem were the key to success in life. However, overdose of this can lead to entitlement. Read more about entitlement below. Healthy doses of self esteem needs to be balanced by healthy doses of self awareness and reality checks.

Entitlement : People who feel entitled view every occurrence in their life as either an affirmation of, or a threat to, their own greatness. If we have problems that are unsolvable, our unconscious figures that we are uniquely special. Somehow unlike others, the rules must be different for us.

If you have a problem, chances are more people have had it in the past or will have it in the future. It just means that you may not be as special as you thought. Your problems may not be priviledged in severity or pain.

How do you balance between self-worth and entitlement and not feel like suffering/entitled ?
The answer is mindset. Do you have a mindset of resilience or endurance ?

Endurance and resilience sound very similar, but are very different. Endurance is short term and resilience is long term.

If resilience is lively, challenging, bouncy and full of flexibility; endurance is characterised by stiffness, survival, cutting off from oneself to get through it. When we endure, we keep going heads down through difficult situations and heavy workloads, sacrificing sleep, activities that we enjoy, relationships with others and self care promising that when this current storm is over, we will rest, connect, get fit, eat healthily. But what happens?

As you have read in the previous posts, life is about moving from one milestone to another. It is about solving a new set of problems. Once you solve a set, there is another set at home or at work which demands you endure again. So the cycle starts again.

Chapter Summary : The subtle art of not giving a F* : Happiness is a problem

Happiness is a problem

1. The first noble truth of buddhism is that life itself is a form of suffering. I think the author here combined buddhism with recursion. The author here says acceptance is the end, the pursuit of the end is another form of suffering. 
2. The rich suffer because of their riches, the poor suffer because they are poor.
3. Happiness is not an algorithm, it is not an end goal. It is a process and a journey. Hopefully the view along the road is what you enjoy. Dont go to the mountains if you like the ocean. 

Reward vs Process

1. You have to figure out whether you are fascinated by the reward. If so you cannot have the reward without the struggle. You cannot choose the result without the process. 
2. People who enjoy long workweeks and the politics of the corporate ladder are the ones who fly to the top of it. 
3. This is not directly about willpower or "no pain, no gain". There might be tradeoffs, you may like the reward so much that you are willing to bear the process which you may not enjoy as much. This is not just corporate america : college students when doing branch selection, often make such tradeoffs. Eg : I am great at chemistry, but I am ok studying computer science because chemistry may not pay the bills. Applogies for chemistry majors reading this, this is not to cherrypick on you, but to explain the concept of tradeoffs. Willpower and grit comes in that whether you are ready to pull through on the process. That is where the sacrifice comes into play. 
4. Its a never ending upward spiral. If you feel that at any point you are allowed to stop climbing, then you are missing the point. The joy is the upward climb itself. You reach milestones on the way : get into top school, graduate top of class, get the top job , get the first promotion, become a manager, become a director. At each step, the problems you are solving are different. What got you here to milestone n, wont get you to the next milestone n+1. So your problems will change, but that doesnt mean they will get easier. The nature of the problems you are solving will change. It is all about whether you enjoy solving those problems or not. It will also depend on which milestone you get to. For example : if you think you are a great engineer and hate management, you will not like the problems that managers have to solve day to day and that kind of job wont scale. That is where matching your milestones with the problems you like to solve matter. 
5. 
   

Chapter summary : The subtle art of not giving a F* : The feedback loop of hell

The feedback loop of hell

1. The feedback loop of hell
• You are anxious about being anxious of something all the time
• You are angry about being angry too quickly all the time
• You are worried about being worried about something all the time
2. Social media today has bred a whole generation who believe that these negative feelings on anxiety, anger and worry are totally not ok. The feedback loop of hell has become a borderline epidemic
3. We feel bad about feeling bag
4. The desire for positive experiences is itself a negative experience and the acceptance of a negative experience in itelf is a positive experience.
5. Countercultural philosopher Alan Watts calls this the law of reverse effort or the backwards law: trying to make everything right often causes things to go wrong. Watts writes, “When you try to stay on the surface of the water, you sink; but when you try to sink, you float.” Security is an illusion. It is only when we acknowledge that insecurity is an inevitable aspect of life that we cease to fear it.
6. The avoidance of suffering is a form of suffering
7. The avoidance of struggle is a form of struggle

How big tech makes money ?

How Big Tech Makes Money

Let’s dig deeper, and see the differences in how these companies generate their revenue.

You are the Customer

In the broadest sense, three of the tech giants make money in the same way: you pay them money, and they give you a product or service.

Apple (Revenue in 2018: $265.6 billion)

• Apple generates a staggering 62.8% of its revenue from the iPhone
• The iPad and Mac are good for 7.1% and 9.6% of revenues, respectively
• All other products and services – including Apple TV, Apple Watch, Beats products, Apple Pay, AppleCare, etc. – combine to just 20.6% of revenues

Amazon (Revenue in 2018: $232.9 billion)

• Amazon gets the most from its online stores (52.8%) as well as third-party seller services (18.4%)
• Amazon’s fastest-growing segment is offline sales in physical stores
• Offline sales generate $17.2 billion in current revenue, growing 197% year-over-year
• Amazon Web Services (AWS) is well-known for being Amazon’s most profitable segment, and it counts for 11.0% of revenue
• Amazon’s “Other” segment is also rising fast – it mainly includes ad sales

Microsoft (Revenue in 2018: $110.4 billion)

• Microsoft has the most diversified revenue of any of the tech giants
• This is part of the reason it currently has the largest market capitalization ($901 billion) of the Big Five
• Microsoft has eight different segments that generate ~5% or more of revenue
• The biggest three are “Office products and cloud services” (25.7%), “Server products and cloud services” (23.7%), and Windows (17.7%)

The remaining tech giants charge you nothing as a consumer, so how are they worth so much?

You are the Product

Both Alphabet and Facebook also generate billions of dollars of revenue, but they make this money from advertising. Their platforms allow advertisers to target you at scale with incredible precision, which is why they dominate the online ad industry.

Here’s how their revenues break down:

Alphabet (Revenue in 2018: $136.8 billion)

• Despite having a wider umbrella name, ad revenue (via Google, YouTube, Google Maps, Google Ads, etc.) still drives 85% of revenue for the company
• Other Google products and services, like Google Play or the Google Pixel phone, help to generate 14.5% of total revenue
• Other Bets count to 0.4% of revenue – these are Alphabet’s moonshot attempts to find the “next Google” for its shareholders

Facebook (Revenue in 2018: $55.8 billion)

• Facebook generates almost all revenue (98.5%) from ads
• Meanwhile, 1.5% comes from payments and other fees
• Despite Facebook being a free service for users, the company generated more revenue per user than Netflix, which charges for its service
• In 2018 Q4, for example, Facebook made $35 per user. Netflix made $30.

So while the tech giants may have many similarities, how they generate their billions can vary considerably.

https://www.visualcapitalist.com/how-tech-giants-make-billions/

AWS by the numbers

To understand the true scale of AWS, you need to look at the numbers.

• AWS has over 1 million active users in 190 countries
• AWS has 5x more deployed cloud infrastructure as their next 14 competitors combined
• Each day, AWS adds as much infrastructure as they used to run in total 7 years back
• Amazon S3 is designed to deliver 99.999999999% durability and scale past trillions of objects worldwide
• AWS partner, Netflix, accounts for up to one-third of Internet traffic during peak usage times
• AWS accounts for 41.5% of the public cloud market, bigger than Microsoft, Google, Rackspace, and IBM combined

References

1. https://www.visualcapitalist.com/stats-amazon-dominance-cloud/

Interesting data points and visualizations

1. Google is the undisputed top website in nearly every country in the world. In fact, Alphabet’s 11 domains in the top 100 ranking – including YouTube and a number of international versions of Google – racked up an impressive 90 billion visits in a single month. - Ranking the top 100 websites in the world. From the investment point of view, this may ranking may not reflect revenue ordering as the world has already gone mobile first and this ordering may not map 1:1 in the mobile first world. 
2. California is the largest economy today – it has a state GDP of $2.6 trillion, which is comparable to the United Kingdom. - Animation of 20 largest state economies in the US. 
3. The United States boasts 41% of the world’s millionaires, but it’s clear that the fruits of labor are enjoyed by only a select group—average wealth ($403,974) is almost seven times higher than median wealth ($61,667). This growing inequality gap knocks the country down to 18th place for median wealth.Countries wealth per capita