Author: diophung

Hacker Laws Every Software Engineer Should Know

One AWS senior principal engineer shared this Github repo with us:  https://github.com/dwmkerr/hacker-laws ([PDF], [Podcast]). The compilation of “Hacker Laws” provides essential guidance distilled from decades of experience. Whether you’re designing a distributed system, optimizing performance, or leading a team, these principles offer valuable frameworks for approaching common challenges.

 

It’s a really good read. Here is the quick summary:

1. Amdahl’s Law

This law emphasizes the diminishing returns of parallel processing. While adding processors may improve system performance, the non-parallelizable portion of your program will limit the overall speedup. For engineers, this serves as a reminder to analyze the structure of the tasks before throwing hardware at the problem.

Key takeaway: Focus on optimizing the bottlenecks before investing in parallelization.

2. Brooks’ Law

The more people you add to a late project, the later it becomes. Brooks’ Law is a cautionary principle that illustrates the inefficiencies created by increased communication overhead and the learning curve of new team members.

Key takeaway: Adding more developers to a problem will not necessarily solve it faster; consider smarter resource allocation.

3. The Broken Windows Theory

This theory suggests that minor issues, like bad code quality or technical debt if left unchecked, will accumulate and eventually cause more significant problems. Treat your codebase like an environment—keep it clean to prevent long-term damage.

Key takeaway: Fix minor issues before they snowball into more significant, harder-to-manage problems.

4. CAP Theorem

The CAP theorem states that distributed systems can only guarantee two out of three properties: Consistency, Availability, and Partition Tolerance. This is critical when designing distributed applications, as you must prioritize based on your system’s needs.

Key takeaway: Understand the trade-offs in distributed systems and decide which aspects are most critical for your use case.

5. Conway’s Law

Organizations design systems that mirror their communication structures. If your company is divided into isolated teams, expect your systems to reflect this disjointed structure.

Key takeaway: Foster inter-team communication to build cohesive and unified system architectures.

6. The Pareto Principle (80/20 Rule)

In software development, 80% of the consequences come from 20% of the causes. The most critical features or bugs are often concentrated in a small portion of the codebase.

Key takeaway: Focus on the 20% of the work that drives 80% of the results—whether it’s features, bugs, or performance issues.

7. KISS (Keep It Simple, Stupid)

KISS advocates simplicity in design. Overcomplicating solutions makes them harder to maintain and scale. Always aim to solve problems with as little complexity as possible.

Key takeaway: Complexity is the enemy of reliability—choose simple, elegant solutions whenever possible.

8. YAGNI (You Aren’t Gonna Need It)

A principle from Extreme Programming, YAGNI advises against implementing features until necessary. Prematurely coding features that might never be used wastes time and resources.

Key takeaway: Build only what you need today, not what you anticipate needing tomorrow.

9. Linus’s Law

This law states, “Given enough eyeballs, all bugs are shallow.” In an open-source or collaborative environment, the more people reviewing the code, the easier it becomes to identify and fix bugs.

Key takeaway: Encourage code reviews and collaboration to uncover issues early.

10. The Law of Leaky Abstractions

Abstractions are necessary to simplify complexity, but every abstraction leaks some underlying complexity at critical moments. Engineers must be cautious about over-relying on abstractions without understanding the systems beneath them.

Key takeaway: Learn the inner workings of your abstractions so you’re prepared when things go wrong.

Final Thoughts: Understanding and applying these laws can elevate your engineering practices. They provide a lens through which to evaluate your design choices, team dynamics, and problem-solving approach. As the software development landscape evolves, these timeless principles remain crucial for success.

What make a great contractor, or the story of how I learn to spot an A player

The fastest and cheapest ain’t always the best

Disappointed Cricket Fan Meme" Sticker for Sale by simcass | Redbubble

So, back in December last year, I wanted to choose contractors to renovate my kitchen. I had a lot of bidders and ended up with the earliest – basically, to avoid decision fatigue. Let’s call him Joe.

Joe was responsive initially, offered me the most affordable price, and got back to me very fast.

I was hooked.

Little did I know that being consistent and predictable sets A+ contractors apart.

Two weeks into the project, he decided to disappear for two days. No text. No call. No nothing.

Communication blackout.

He promised to finish the project before Christmas – and as of now, near the end of January, we’re still living without a kitchen.

I regretted hiring him.

The most consistent and predictable player, win.

The Best Handyman Near Me

Enter the story about Jim.

Jim is another contractor I liked and helped me with various projects around the house: bookshelves, closets, you-named-it.

Was he the first? No.

Was he the cheapest? Not likely.

Was he the most consistent and predictable? Boy, oh boy, yes, absolutely yes.

Jim said he’ll be there at 9 am to start working. At 8.55 am, the bell rang, and he was there.

Jim would work continuously for 12 hours to complete the project on time. He told me it would take this much time up front, and he was right to a quarter of the hour. He communicated frequently with me about the progress and consulted me when anything was off.

He delivered the work beautifully.

I’d gladly hire Jim again for any future project. Not Joe.

——–

That’s the consistency and predictability that set A player apart: being the fastest or cheapest might only get you going. You need to be consistent and predictable to win and deliver amazing results.

That’s it. Now is my time to return to my kitchen.

 

On Amazon cultures and practices

I joined AWS a while back, as an engineering manager. I had the wonderful opportunities to learn from Amazon unique cultures and wanted to write something about it. Here is my first attempt to share a bit of Amazon culture, came right out of my onboarding.

  1. Customer Obsession

Every company on earth will say that.

Everyone will say: customer is at the center of everything we do.

But Amazon stands out by being embedding this leadership principle into every day works: want to make a product proposal? Write an PRFAQ, like a press release, describe how customer would use and benefit from this product when it’s release.

On another aspect, I remembered this scenario during my manager onboarding :

“If an AWS customer accidentally incurred a $16,700 bill, how much the customer can claim back?

a) 50%
b) 65%
c) 80%
d) 100%”

This is a real-life example.  Ans: You got the right answer, didn’t you? It’s d) – 100% !

  1. Dive Deep

Amazonians circulate this serious joke: “In God we trust, everyone else bring data”. Not that we don’t trust each other, but we value the factual facts as much as believes.

One outstanding example is COE (Correct of Error) review: we listed detail timeline, action we took, impact we observed and ask ourselves 5-Whys, peeling layer of the onion until we can find root cause.  More often than not, the root cause and the best solution is not trivial, but the effort worth it.

Here is one example on the importance of asking 5-whys:

Problem: One of the monuments in Washington D.C. is deteriorating.

Why #1 – Why is the monument deteriorating?  

    • Because harsh chemicals are frequently used to clean the monument.

Why #2 – Why are harsh chemicals needed?

    • To clean off the large number of bird droppings on the monument.

Why #3 – Why are there a large number of bird droppings on the monument?

    • Because the large population of spiders in and around the monument are a food source to the local birds

Why #4 – Why is there a large population of spiders in and around the monument?

    • Because vast swarms of insects, on which the spiders feed, are drawn to the monument at dusk.

Why #5 – Why are swarms of insects drawn to the monument at dusk?

    • Because the lighting of the monument in the evening attracts the local insects.

So the best solution actually is:  Change how the monument is illuminated in the evening to prevent attraction of swarming insects.

3. Perculiarity 

Yes, you read it right. Amazon values perculiarity – doing thing in a different way, sometime, strange way. Amazonians are proud to be different and doing things in unique way.

Ever go to large meeting and nodding off? Amazonian fix that with a brilliant simple idea: use a spin wheel in meeting, randomly pick one of the audience to participate. Everyone suddenly stays engaged!

Going to a review and not sure what everyone is talking about? Well, Amazonians counter that by spent silent time in the beginning of a meeting, to read the materials.

These are indeed perculiar practices. And they works, beautifully !

That’s a wrap for now. I’ll love to share more cultural practices, as I learn more from my journey.

 

References

[1] LP = Leadership Principles. They appear in almost all everyday conversations.

[2] PRFAQ = Press Release FAQ – a document to write before building any product or service, describing how we would talk about it when it is released: how it will benefit customers, what customers will be interested in.

[3] COE = Correction of Errors – a detailed, nameless post-mortem with detailed documentation on history, timeline of the event, impacts, root cause, action items and proposal.

Leading Through Uncertainty Time

Leading Through Uncertainty Time

This came in my mailbox from Harvard Business Review. It resonated greatly with my current context: dealing with new scopes, zooming in on net new focuses, and tackling greater challenges.

Sharing it here as a way to remind my future self. You can read the whole article on HBR here.

Facing Uncertainty – It’s All About Mindset
Uncertainty is unavoidable. As a manager, you need to be prepared to lead your team through murky waters, but doing so requires getting in the right mindset yourself. Here are six tips to help you shift your perspective:

1. Embrace the discomfort of not knowing. Move from a know-it-all to a learn-it-all mindset. You don’t need to have all the answers.

2. Distinguish between “complicated” and “complex” issues. They require different solutions.

3. Let go of perfectionism. Instead, aim for progress, expect mistakes, and recognize that you have the ability to continually course correct as needed.

4. Resist the urge to oversimplify and come to quick conclusions. Take a disciplined approach to understand both the complexity of the situation and your own biases.

5. Don’t go it alone. Connect with your peers who have their own set of experiences and perspectives to draw from.

6. Zoom out. Taking a broad, systemic view of the issues at hand can reveal unexamined assumptions that would otherwise be invisible

12 factors of modern SaaS

TL;DR: it’s a guide with best practices for any engineer who builds & deploys SaaS, based on experience working and scaling many apps on Heroku platform.

Introduction

In the modern era, software is commonly delivered as a service: called web apps, or software-as-a-service. The twelve-factor app is a methodology for building software-as-a-service apps that:

  • Use declarative formats for setup automation to minimize time and cost for new developers joining the project;
  • Have a clean contract with the underlying operating system, offering maximum portability between execution environments;
  • Are suitable for deployment on modern cloud platforms, obviating the need for servers and systems administration;
  • Minimize divergence between development and production, enabling continuous deployment for maximum agility;
  • And can scale up without significant changes to tooling, architecture, or development practices.

The twelve-factor methodology can be applied to apps written in any programming language and use any backing services (database, queue, memory cache, etc.).

I. Codebase

One codebase tracked in revision control, many deploys. One app, one codebase. Each component maps to one app (or service) in a distributed system.

One codebase maps to many deploys

II. Dependencies

Explicitly declare and isolate dependencies. Never depends on implicit system-wide packages. Declare all dependencies completely and exactly. Use a dependency isolation tool (virtualenv for Python) to prevent leaks or dirty dependencies.

III. Config

Store config in the environment. Dev, staging, production environment should have different configs. Separate config from code.

IV. Backing services

Treat backing services as attached resources, i.e., local and third-party services can be used interchangeably. Resources can be attached/detached at will.

A production deploy attached to four backing services.

V. Build, release, run

Strictly separate build and run stages. Restrict changes to the code at runtime.

Code becomes a build, which is combined with config to create a release.

VI. Processes

Execute the app as one or more stateless processes. Processes are stateless and share-nothing. This will allow the app to scale horizontally. Any data that needs to persist must be stored in a stateful backing service, typically a database.

VII. Port binding

Export services via port binding. The app is completely self-contained and does not rely on runtime injection of a webserver into the execution environment to create a web-facing service. The web app exports HTTP as a service by binding to a port and listening to requests coming in on that port.

VIII. Concurrency

Scale-out via the process model. In the twelve-factor app, processes are a first-class citizen. Processes in the twelve-factor app take strong cues from the Unix process model for running service daemons. Using this model, the developer can architect their app to handle diverse workloads by assigning each type of work to a process type. For example, HTTP requests may be handled by a web process, and a worker process handles long-running background tasks.

Scale is expressed as running processes, workload diversity is expressed as process types.

IX. Disposability

Maximize robustness with fast startup and graceful shutdown. App processes are disposable and can be spun up / spun down easily. App on shutdown should exit gracefully, such as returning job to the queue or closing existing connections upon receiving SIGNTERM.

X. Dev/prod parity

Keep development, staging, and production as similar as possible.

XI. Logs

Treat logs as event streams. The app never concerns itself with routing or storage of its output stream. On localhost, it can be just stdout. On a production system, the streams are captured by a log system.

XII. Admin processes

Run admin/management tasks as one-off processes.

One-off admin processes should be run in an identical environment as the regular long-running processes of the app. They run against a release, using the same codebase and config as any process run against that release. Admin code must ship with application code to avoid synchronization issues.

 

Credit:https://12factor.net/

Interviewing tips for interviewers

As a manager, one of the most important tasks is to hire the right talent for the team.

Well, it might be the most important one.

Yet almost everyone dreads interviewing.

How can we avoid asking the type of “tell me about yourself” question out of habit? Or how we can get to the real point instead of asking “tell me about a time you struggle”?

Well, after having two coaches, being mentored by 4 industry veterans, 15 years in this industry, conducting over 110 interviews, I had a few tips for effective interviewing to share.

Technical screening round

The goal of the screening interview round is to ensure we have the right candidate when it comes to onsite interview rounds. The screening interviewer should filter out unqualified candidates as soon as possible. Your time is valuable, and so is everyone on your team.

With that, here are things you should do:

  • Do the homework:
    • Research candidate before the interview: use your 360-degree lens, dig in LinkedIn, scan the CV to find patterns: is she a fast learner? Is she pushing her out of her comfort zone? Was she a team player or a solo? See if she can show her potential to grow in this position. Don’t ask superficial questions such as “tell me about yourself”.
  • Go hard on the technical side with a nice tone:
    • Don’t settle on easy questions. It won’t help. Remember: “A player attracts A player, B player attracts B, C, and even F player”.
    • Push the candidate until she said, “I don’t know”. Great people know their limits, they don’t try to show that they know everything. You need to push to see what her boundaries are to set her up for success if you hire her.
    • Don’t stop at the first solution:  A solid candidate always tries to improve, even if she found a solution. She would find a working solution, lean on that, improve for certain dimensions. Need to trade memory for speed? Or readability over coding speed? Ask the candidate if the algorithm can be further optimized in terms of time & memory usage? Will that work with +100 million items, or with just 1MB of RAM?
    • Provide assistance and support when the candidate is stuck. Give reasonable hints, coachable talents pick up hints very fast.
  • Be open-minded and look for room for improvements
    • A phone interview is not easy for both sides, if the candidate has trouble understanding, offer help. The goal of the screening interview is to measure candidate problem-solving and communication skills.
  • Make it an open conversation: 
    • The interview doesn’t have to be one-way, and ideally, it should be like an intellectual conversation so give open suggestions, listen and give feedback appropriately.  Don’t impose your opinions and knowledge on the answer: if the candidate chooses Python even though we code in .NET / JavaScript, that’s fine. As long as she demonstrates solid data structure and algorithm expertise, the choice of language and style differences can be ignored.
  • How to start a Problem Solving challenge:
    • Start by asking what’s the general algorithm? Does it “sound” like a solution, is it working?
    • Start to draft an optimal algorithm then proceed to implement

Onsite interview round

Firstly, the onsite round is to ensure the candidate will be a good culture fit, with solid communication skills. Secondly, it is to have a broader assessment of the technical skills. It is also about presenting our team, our culture, our people. It is to find a colleague that we’d love to work with on a daily basis.

Onsite interviewing is a chance to leave good impressions on the candidate so that even if she won’t get the job, she will be an ambassador for us. Remember interviewing works both ways: candidates evaluate interviewers at the same time so find your way to create an uplifting experience.

  • Sync up beforehand:
    • Discuss with the hiring committee the type of questions and topics which each interviewer will cover.
    • Try not to have multiple interviewers interviewing on the same topic – unless it is critical for the job. Your hiring committee should be representative so that each person can probe the candidate on a dimension.
  • Ask open-ended questions: 
    • Ask a problem that has multiple solutions so that we can see how the candidate handles ambiguity and unknowns.
    • Aim for the problem that the candidate never solved before but can be solved with additional data and help.
  • Separate well-practiced answers from real ones:
    • 5-whys: keep asking why. A great answer is one that can go deep through multiple layers of that onion.  Sometimes, great people will throw their hands in the air and say “I don’t know why”, but by then you would have enough data to consider.
  • Share feedback as soon as possible: 
    • Ideally, once the hiring committee finishes interviewing, everyone should meet and provide feedback when the memory is still fresh. Every hour passing by, the quality of the feedback degrades.
    • The trick to avoiding herd mentality is to have everyone put down their vote before they meet: it’s either Yes or No – do not accept Maybe! If one needs to switch the vote, there must be really strong reasons.
  • Be professional, move quick
    • In case the hiring committee cannot meet soon, keep the candidate posted about when she can expect the output. If the team can meet and agree this is the right talent, make a case with your decision-makers.

Well, thank you for reading this far.

This post is by no means a complete list, it rather serves as a starting point and hopes it spark your interest in interviewing. And the Aha! feeling when you find that great talent? It’s totally worth it!

Have other opinions? I’d love to hear from you in the comment section.

From F to A+

TL;DR: this post is about how I improved my site ssllab.com HTTPS rating from F to A+ .

Make HTTPS great again.

Not all HTTPS-enabled sites are created equally, so welcome to the Internet of broken protocols and pardon my usage of a political phrase.

When I set up this site long ago, it was running on VPCs with many components which with today’s standards, were considered vulnerable: OpenSSL 1.1.0g, PHP5.4, Apache2.1 on Debian 7. You named it.

That’s why ssllabs.com used to rate my site as F:

Before

file

After

It’s rated A+, and I’m pretty proud of it.

file

How?

Here are the detailed steps on how I improve my site security from F to A+ in few simple steps.

1. First thing first: bye-bye to outdated OpenSSL

The previous version of openssl on Debian was suffering from these 2 critical vulnerabilities: SSL Pulse (CVE-2014-0224) & Padding Oracle (CVE-2016-2107). That’s the reason why SSLLab report an F.

The fix was fairly simple: upgrading OpenSSL.

sudo apt update && sudo apt upgrade openssl libssl-dev

After that, check the version:

openssl version
OpenSSL 1.1.1d  10 Sep 2019

2. No more outdated ciphers

SSLLabs also reported another two issues which cap the grade at B:

  • “This server accepts RC4 cipher, but only with older protocols. Grade capped to B”
  • “This server does not support Forward Secrecy with the reference browsers. Grade capped to B.”

Ok, onto finding Apache2 config files:

# assuming Apache2 is at /etc/apache2 
grep -i -r "SSLEngine" /etc/apache2
/etc/apache2/sites-available/default-ssl.conf:   SSLEngine on
/etc/apache2/sites-available/diophung.com.conf:  SSLEngine on

Here we go, the config files are default-ssl.conf and diophung.com.conf. From there, I decided to remove RC4 due to its flaws, and then enable Forward Secrecy in the config files:

SSLProtocol all -SSLv2 -SSLv3
SSLHonorCipherOrder on
SSLCipherSuite "EECDH+ECDSA+AESGCM EECDH+aRSA+AESGCM EECDH+ECDSA+SHA384 EECDH+ECDSA+SHA256 EECDH+aRSA+SHA384 EECDH+aRSA+SHA256 EECDH+aRSA+RC4 EECDH EDH+aRSA RC4 !aNULL !eNULL !LOW !3DES !MD5 !EXP !PSK !SRP !DSS !RC4"

After that, restart Apache2 and recheck

apachectl -k restart

3. Why stop at A when you can get A+?

At this step, ssllabs rated the site as A, which is pretty good result. But I figured I can get to even better result A+ and being me, I wouldn’t stop. So the next step, is to enable HSTS:

I opened up the Apache2 config files and add this HSTS header:

<VirtualHost diophung.com:443>
Header always set Strict-Transport-Security "max-age=63072000; includeSubdomains; preload"
</VirtualHost>

Feel free to read up about HSTS if you’re curious.

Voila!

After everything, the site is now rated as an A+ result. I’m pretty happy about it and a bonus is that the site also loads 45% faster. So, strongly recommend you give it a try: https://ssllabs.com and let me know what score do you have?

 

Stanford LEAD, an amazing journey

Stanford GSB, 2021

Over, but not done

Yes, it is here: this week I received my Stanford LEAD graduation certificate in my mailbox after a year-long journey.

After 1 year, 9 courses, 10 teams, 83 submissions, and hundreds of self-research hours, I can proudly wrap up another chapter in my life-long learning journey.

How it started

It was in August 2020, the COVID-19 pandemic has been going on for over a year, I decided to turn this challenging time into a memorable time. At work, I was leading my teams with a net new initiative, a critical mission to help my company grow 5X over the next 3 years. At home, I was expecting a new baby and at the same time, my 2-year-old son was ready to go to preschool. We’re also moving to a new home.

One might say there was never a busier time.

But I did it. I chose to go to Stanford. One month after submitting my essays, references, and video presentation, I received the Stanford welcome letter.

Reflection on the course

Throughout the year, I had the opportunity to meet Stanford GSB’s world-class faculty. From renowned professors, inspiring course facilitators to amazing fellow LEADers – leaders of their own organizations, all have been very welcoming. It’s been an absolute pleasure to have the opportunity to learn, share, and practice all aspects of leadership.

The contents were excellent, with each course being designed to be very interactive. The case studies were fantastic with relevant industry examples and many were from Harvard (yes, HBR articles are weekly must-read). I must say I loved the readings and case study, but not so much for written submissions 🙂

The course structure was pretty flexible with offline readings and 1-hour Zoom call every week with professors and course facilitators (CF). Our CFs were wonderful partners and many of them were in fact LEAD alumni. I was truly humbled to have my coaching sessions with many of them.

Fun fact: each Stanford LEAD cohort is given a unique name representing the GSB spirit. In the past, we have had names such as Vanguards, Explorers, Pathfinders – mine is Navigators. It meant so much when the whole world was navigating uncharted water with the COVID-19 pandemic.

Final thoughts

Being a life-long learner, I’d wholeheartedly recommend Stanford LEAD to anyone who aspires to be a leader in your organization and considering. To help with the course selection, I will share the courses I took, together with my experience in another blog post.

Here are some excerpts for a preview:

  • Principled and Purposeful Leadership
    Rank: A
    Leadership lessons through self-reflection, looking inward, looking outward, defining your own values, mission, then defining an execution plan for your mission within the organization. Executive coaching sessions available.
  • Critical Analytical Thinking
    Rank: A+
    Frameworks for thinking logically, realizing biases and deriving reasonable conclusions, plenty of practicing with team and debates, excellent reading materials & examples on how some legendary leaders in the industry made their decisions.
  • Financing Innovation: The Creation of Value
    Rank: A-
    Corporate finance, financial statements (P & L, cash flow, annual reports), method to calculate WACC (Weighted Average Cost of Capital), understanding startup funding series (pre-money, post-money value).
  • Strategic Leadership
    Rank: B+
    General leadership strategies, defining a firm’s core strengths and advantages.
  • Communicating with Impact
    Rank: A+
    Solid techniques and strategies, applicable frameworks for effective communication.
  • Decision Making
    Rank: A
    Frameworks and tools for well-rounded, sound decision making process with imperative and data-driven approaches.
  • Customer Experience Design – A Neuroscience Perspective
    Rank: A-
    Put customers first, see through their lens, leverage the X framework to convert customers from low → high-energy engagement.
  • Persuasion: Principles and Practice
    Rank: A+
    Superb psychological insights & comm strategies. Simple yet effective examples through leadership stories.
  • The Innovation Playbook
    Rank: A
    Imagine you’re a startup founder with a problem & an idea: these are the steps to take your product from concept to POC to launch.

Support for Keyboard Shortcut to Close Chrome tabs

A quick guide to fix issue with my “Keyboard shortcuts to close Chrome tabs” extension

Symptom

Unable to use the shortcuts anymore, or unable to open the Options page:

error message

Solution

Right click on Chrome menu bar > Manage extensions. Look for the name “Keyboard shortcut to close Chrome tabs” (yes, I could choose a shorter name), make sure it is not corrupted. Here is an example of a corrupted extension:

file

If it is corrupted, click the Repair button. Expected to see:
file

Now reload Chrome and retry.

Root cause

An error while Chrome was trying to restore your profile settings, or a corrupted file system might cause this issue. In my case, it was when I switched from my Mac to my Windows machines, Chrome failed to restore all the installed extensions.