Blog

Check the disk space consumption of a Linux directory

Today, one of my virtual systems ran out of disk space. To find the root cause I needed to find the directory that was filling up. Using the right parameters, the du command returns a list of directories and their disk space consumption.

du -hc --max-depth=0 /var

For example:

root@DB152:/# du -hc --max-depth 1 /var
704K    /var/tmp
4.0K    /var/mail
23M     /var/log
1.1M    /var/backups
4.0K    /var/local
3.2G    /var/lib
20K     /var/spool
734M    /var/cache
4.0K    /var/opt
3.9G    /var
3.9G    total

Removing files of a certain type recursively:

find . -name "*json.log" -type f -delete

Clearing (system) log files:

truncate /var/log/* --size 0

Prioritisation

The goal of prioritisation

We always have more ideas, but never seem to have the resources to execute them. Therefore, it is important (both in private and professional life) to carefully cherry-pick a couple of ideas and go all-in. However, keep in mind: it’s never wrong to pivot.

What are we prioritising?

Depending on the industry you’re in you might be prioritising different things. In my work I prioritise a software backlog, but in my private live I (consciously) prioritise in the way I’m building my house or (unconsciously) the things that intrigue me most. In all of these situations we should ask ourselves: what is the goal that I want to achieve? Without having a clear (and preferably measurable) goal in mind, you cannot prioritise.

If you’re interested in the use of goal-models and how to apply these in requirements engineering, I can highly recommend the article Reasoning About Alternative Requirements Options.

Frameworks

After you’ve defined your goals and what you want to prioritise, you should understand the available frameworks. Note that I didn’t write you should select a framework. I don’t believe there is a one-size-fits-all approach to this. Selecting the right model depends on your personality, way of working, and the context you are prioritising. The UX Collective wrote an article that lists the available prioritisation frameworks and explains the different categories.

Source: UX Collective, How to choose your Product Prioritization Framework

Let me introduce you to two of these frameworks: KANO and RICE. This combination works well for me personally. They are pragmatic, easy to explain, and easy to implement.

RICE

RICE was developed by Intercom and is an abbreviation for:

  • Reach: how many of my users will find this thing useful?
  • Impact: how much impact will this thing have on the metrics we have defined that allow us to reach our vision?
  • Confidence: how confident are we on our estimations for reach, impact, and effort?
  • Effort: how much do we need to invest to get the thing in a state that users will start using it?
Source: Productplan, what is the history of the RICE scoring model?

You should define a range for these variables before you assign values o them. Then, you can easily calculate the RICE score by R * I * C / E.

KANO Model

John Vars applied Maslow’s hierarchy of needs to product development. He came up with three layers:

  1. The foundational layer includes functionality that users are not able to live without.
  2. The value proposition layer makes a user successful or it solves a pain.
  3. The growth layer really excited a user by solving a problem that the user wasn’t even aware of. This should set you apart from other solutions as well.
Source: John Vars, the product hierarchy of needs

The KANO model uses a similar approach: it’s goal is find a balance between things that users are disappointed on if they are missing (threshold attributes) and things that excite users (excitement attributes). When excitement attributes mature their implementation depth grows and they slowly move right, into the performance attributes.

Source: Mindtools, Kano Model Analysis

Practically working with RICE and KANO

RICE results in a number. You can easily capture this in a spreadsheet and sort using the highest number. KANO results in a classification. You can easily create a simple spreadsheet to capture the things you want to prioritise. Assign the value and you get an idea of what should be important to you.

Example of the implementation of the RICE and KANO model in a simple spreadsheet.
Example of the implementation of the RICE and KANO model in a simple spreadsheet.

I recommend that you always interpret the output of the models. Ensure the theoretical prioritisation makes sense in practise.

Implementation and validation

These models require a bit of time to understand and apply. I would suggest to implement them in two steps:

  1. Use them based on experience and a couple of explicit assumptions. This immediately gives you an idea of where you stand.
  2. Once you got the hang of it and found a good way of working, start to validate the assumptions and experience with the users of the thing you are creating. This probably lengthens the prioritisation process. You need to decide if this time is worth it.

Transparency

Taking the time to explain how you came to a decision is just as important as communicating the decision itself. Both RICE and KANO allow you to visually explain how you prioritised the items you are going to work on.

The Pareto principle in Power BI

Product Managers love data. Therefore, being able to create your own reports to track the KPIs relevant for your products is really useful. One of the KPIs you might run into relates to the Pareto principle, for example, 80% of the revenue needs to come from 20% of the customer base.

The Pareto principle

I’m going to refer you to the wikipedia for the extensive explanation. I will give you a couple of examples:

  • 20% of the customers should account for 80% of the revenue.
  • 20% of the richest persons in earth account for 80% of the income.
  • 20% of Italy’s population owned 80% of its land.

The Pareto principle in Power BI

We take a couple of steps to get this visualised in Power BI:

  1. Prepare the data structure.
  2. Adding a rank to the data based on a specific category.

As an example I’m using the KPI that 80% of revenue should come from 20% of the customers.

Data preparations

I recommend you to create a table to put the data you want to visualise on the screen. This makes it easy to validate the results of the measures you create.

Adding a table to a Power BI report.
Adding a table to a Power BI report.

You need two types of data to visualise the Pareto principle in Power BI:

  1. Category: this can be, for example, a customer or country.
  2. A number: this can be, for example, the sales of a customer or country.

Ranking your customers

You first need to find the top 20% of your customers. You should create a measure that calculates the rank of the customer depending on the sales value.

Rank = RANKX(ALL(Sales[Customer]), CALCULATE(SUM(Sales[Sales_Value])))

When you add the Rank measure to your table and sort it by sales value, the customer with the highest amount of sales should have rank 1.

Cumulative sales

Now that we have ranked your customers, we want to calculate the cumulative sales. This measure cumulative sales of the customer with rank 1 is the sum of the sales of customer 1. The cumulative sales of customer 2 is the sum of the sales of customer 1 and customer 2.

(use SHIFT+ENTER to create a new line in DAX).

Cumulative_Sales_Value = 
var currentRank = [Rank]

RETURN SUMX(FILTER(ALL(Sales[Customer]),[Rank]<=currentRank), CALCULATE(SUM(Sales[Sales_Value])))

Total sales

To calculate the stake of the cumulative sales of a customer in the entire revenue you need to calculate the total sales value. The ALL function applies the selected filters.

Total_Sales = CALCULATE(SUM(Sales[Value]), ALL(Sales[Customer]))

The last measure you need to create calculates the cumulative sales (in % of total sales). You will use this measure to create the line in the visualisation of the Pareto principle.

Cumulative_Sales_Perc = [Cumulative_Sales_Value] / [Total_Sales] * 100

Create a line and clustered column chart

Now you are ready to add the measures into a line and clustered column chart.

Adding a line and clustered column chart to your PowerBI report.
Adding a line and clustered column chart to your PowerBI report.
Assigning the Customer, Sales_Value, and Cumulative_Sales_Value_Perc to the line and clustered column chart.
Assigning the Customer, Sales_Value, and Cumulative_Sales_Value_Perc to the line and clustered column chart.

The result:

An example of the result of visualising the Pareto principle in Power BI.
An example of the result of visualising the Pareto principle in Power BI.

Automatically push .NET libraries on NuGet

Getting the source code of my home automation project on Github was a first step. Now, I’m ready to post some components on NuGet as well. This makes it easier to manage the packages and reduces the dependencies between my Visual Studio projects.

I ran into a challenge though: how could I easily upload my projects to NuGet without having to use the command line? Luckily, Visual Studio has something called Post-build event command line.

Post-build event command line; you can find this in the properties / build events tab of your project.

The command put into this block is executed after finishing the build. The only thing you need to do manually is change the package version, otherwise NuGet will rightfully decline the push (the version already exists).

You need to manually update the package version for NuGet to accept the push.

Depending on your directory and project structure you should be able to get the command below up and running. You can easily test this in the Command Prompt, just make sure you browse to the right target directory.

cd bin\Debug && dotnet nuget push *.nupkg -k xxx -s https://api.nuget.org/v3/index.json --skip-duplicate

Of course, you should replace the xxx with the NuGet API key.

Reduce disk space consumption in Windows 10

Windows 10 can be quite hungry when it comes to disk space, especially when the system has been used for several years and has been updated several times. In some situations temporary update files are “stuck” in the C:\Windows\WinSXS directory, which at that time can consume up to 17Gb of disk space. Microsoft has published an article describing the process to clean up this directory in more detail. However, this doesn’t always work as described.

I recommend you to do the following:

  1. Download the latest version of System Cleanup, an old but ad-free program that scans the system for temporary files.
  2. Run the Disk Clean-up (as administrator!) and clean-up the system.
  3. Use Dism, the command line deployment tool, the merge existing update images into the current installation using the commands below. This might take several hours to complete. You might need to repeat them once or twice to take effect.
Dism.exe /online /Cleanup-Image /StartComponentCleanup /ResetBase
Dism /Online /Cleanup-Image /RestoreHealth
Running System Cleanup, an old but ad-free tool that scans the system for temporary files.
Starting Disk Clean-up as administrator. Make sure you tick all necessary boxes for clean up.

These steps helps helped me to free up roughly 12Gb of storage space on a small little tables that only had a 32Gb drive. Its users can now enjoy using it again for the occasional e-mail and card game, without worrying about a lack of disk space and, as a result, missing important security updates.

Publishing an existing .NET (Core) project to Github

I started working on my home automation project in 2020. I started with a simple backup script that copies the source code to a second hard drive, just in case the primary hard drive fails. When my code base started to grow I extended the script to create a (daily) second copy on OneDrive. From a backup perspective this is working fine, but it does not give me versioning and source control. Luckily, Github exists. Microsoft developed a Github extension for Visual Studio.

Make sure you update your Visual Studio installation before you install the extension. You also need to install Git for Windows. Additionally you need a Github account. After that, publishing an existing project to Github is pretty straightforward.

Open the command line and browse to the directory of the project. Execute a git init.

D:\>cd DataMario\Ontwikkeling\IoT\Verhaeg.IoT.Geofencing.Sensor

D:\DataMario\Ontwikkeling\IoT\Verhaeg.IoT.Geofencing.Sensor>git init
Initialized empty Git repository in D:/DataMario/Ontwikkeling/IoT/Verhaeg.IoT.Geofencing.Sensor/.git/

D:\DataMario\Ontwikkeling\IoT\Verhaeg.IoT.Geofencing.Sensor>

Open Visual Studio and copy/paste an existing gitignore file to exclude configuration files that could include passwords.

Then push the project to Git:

Push the project to Git service…

And finally, define the Git repository details.

Visual studio will offer you to create a Github repository.

Once you have published the project on Github you need to maintain it. There are three steps required to upload changes to the Github repository:

  1. Pull: pulls the latest version of the repository to the local development machine.
  2. Commit: commits the code changes to a “snapshot” of your project.
  3. Push: pushes the latest snapshot of your project to the repository.