Strenuous Life

I keep a daily reminder to myself to live life strenuously because for me it is the most satisfying way to live.

I was exposed to this idea by reading Teddy Roosevelt’s speech on this topic, and the idea has come back into my daily thoughts about once every 1-2 months over the last 5+ years. It struck a deep chord. I have only recently fleshed out what it means to me in more detail.

To me it means:

Embrace discomfort. Plan big, uncomfortable events. Run a marathon, learn a difficult skill, go on a big trip. Always live outside the comfort zone.

Maximize the number of uncomfortable, impactful actions each day. Embrace the discomfort of each and every small step along the way to reaching larger milestones. If I’m doing something and I feel like it is easy or I feel comfortable, that’s a red flag.

Push myself beyond my limits every day. Maximize the speed and delivery of discomfort by eliminating hesitation. Always have the pedal to the metal.

Not a day goes by when I don’t fall into bed feeling totally spent[1]. One of the most joyful feelings is when, at the end of the day, I’m so exhausted that I feel like I can’t think another thought. On days like that, I know I grew a lot.

For me, the strenuous life is not restricting but liberating. I feel like my personal and professional growth is maximized by living this way–squeezing every drop out of my life. After a while, embracing discomfort became comfortable and I started to feel uncomfortable with comfort.

[1] To clarify, I place very high emphasis on sleep. I’m very protective of my sleeping time and rarely sleep less than 8 hours. Sleep deprivation for even just 1 day makes my effectiveness and efficiency go way down in most areas. I think that living strenuously actually improves sleep quality because when I go to bed I’m actually really tired, fall asleep almost instantly, and sleep soundly all night.


Silence

High quality thinking is rare but necessary to accomplish ambitious goals. Extended periods of silence are one of the best ways to produce high quality thinking, but people are unlikely to accidentally spend even 2 minutes in silence (e.g. zero external inputs to your brain). The world is too loud. Silence needs discipline at first, but the benefits become clear quickly and a natural desire for silence emerges.

Challenge for you right now

Look at a tree for 30 minutes.

There is no goal except to not have a goal. Calculating thought gradually gives way to connecting thought. You might be surprised by how powerful your brain is when you give it time to process things in silence. Insights bubble to the surface.

Related quote

“It is not necessary that you leave the house. Remain at your table and listen. Do not even listen, only wait. Do not even wait, be wholly still and alone. The world will present itself to you for its unmasking, it can do no other, in ecstasy it will writhe at your feet.” - Franz Kafka

Rebuke

The word rebuke[1] may make you feel uncomfortable. I view it as a good thing.

The benefits of a good rebuke, well-received are difficult to overstate. It can save years of professional setbacks, interpersonal relationship issues, and even financial loss.

It may not be unwise to explore increasing the number of rebukes between people closest to you.

Person A wronged you. Person B’s behavior negatively effects people around them and they don’t seem to notice. Person A is your friend. Person B is your co-worker, family member.

These scenarios are common and it is unfortunately also common for close friends who want to speak up to decide against it for fear of damaging the relationship.

I think it is worthwhile to acknowledge this dilemma and take action to remove the mental anguish friends may feel regarding their decision to rebuke or not. One way to start along this path is to ensure friends know that they don’t need to worry about upsetting you or damaging the friendship by saying something critical.

However, I don’t think this goes far enough. There are a few other obstacles that need to be overcome:

  1. Thinking the concern is over a matter too small to initiate an uncomfortable conversation
  2. Not paying close enough attention to notice any issues
  3. When issues are noticed, they are not recorded and forgotten or deprioritized

The countermeasures to these pitfalls are to share how to collect specific examples and to assure them that no concern is too small to share. I think it is actually helpful to receive seemingly small and trivial rebukes because it helps me get used to receiving rebukes without reacting in anger or defensiveness.

[1] I considered using other words: correction, constructive criticism, improvement suggestion. I decided to use the word rebuke because it is very specific to the type of scenario that I’m writing about and it discourages sugarcoating.

FATWOAF

Layers of Thought

Thoughts are the lowest layer in the structure from which character proceeds.

Layer # Layer Name
1 Thoughts
2 Feelings
3 Words
4 Actions
5 Habits
6 Character

As the layer number increases, direct modification gets harder. It’s probably impossible to wake up one day and permanently change character traits before going to sleep that night.

If you want change, then go to successively lower layers until you are successful. If it is too hard to change your actions directly, start by changing your thoughts. It is a lot easier to change your thoughts because you don’t have to do anything physically exhausting, time consuming, or mentally straining. You just have to get intentional and consistently think a thought that is outside your normal thought life.

For instance, it might be too hard to consistently run 2 miles per day (layer 4) if you haven’t run for a while. Instead, set a daily reminder on your phone that says, “Lack of exercise makes me go crazy. I cannot think well if I don’t run. Other parts of my life become harder when I don’t run.”

When the reminder pops up, just read it and check it off. You don’t have to do anything but think the thought. The other layers will proceed from your thoughts. The thought will gather enough momentum in your inner thought life, that you will start to feel (layer 2) that you want to run.

I have seen more lasting success in changing my actions with this approach than trying to modify my actions directly. I really think this approach is the key to sustainable self-evolution. I have made several 1% changes using this approach that compound make my life a lot better. It takes time but it’s very reliable.

  • Rewritten and expanded 01-29-2022

Thoughts on Hackerrank's Big Sorting Problem

I enjoy using Hackerrank to sharpen my understanding of fundamental computer science concepts.

I recently solved their “Big Sorting” problem and would like to share my thoughts on it.

I will assume you know understand the problem already. Here is the problem statement in case you’re unfamiliar with it..

The problem seems to be fairly straightforward: sort a list of numbers represented as strings.

It’s easy to write a one liner in Python to accomplish this:

numbers_array = ['9', '1', '3']
sorted(list(map(lambda number: int(number), numbers_array)))

It’s just three easy steps:

  1. Convert each numeric string to an actual int object
  2. Create a list with the newly created ints
  3. Sort the list

This approach seems too easy since the problem only has a ~63% solve rate on Hackerrank. It can’t really be this easy, right?

Unfortunately, this naive approach doesn’t pass several of Hackerrank’s test cases because it’s too slow.

In what cases do you think this approach would be too slow?

It turns out that very large numbers make this approach too slow. I bought one of Hackerrank’s test cases that was failing, and realized that one of the numeric strings in the list of inputs was 988k characters long. That’s a really big number! Thankfully, Python 3’s int class can handle ints that size, but there’s a big catch: it takes a while to instantiate them. To be precise, it took 5.96 seconds to instantiate the 988k digit long number as an int.

The script below takes 6.16 seconds to execute, so about 96.8% of the execution time was taken up instantiating that massive int.

numbers_array = ['9', '1', '3', '9'*998000]
sorted(list(map(lambda number: int(number), numbers_array)))

Python 3’s int class just wasn’t created to handle numbers of that length. But the Decimal class handles it much faster. To be precise, Decimal('<insert 988k digit long number here>') only takes 0.06 seconds. This is ~98.7% faster than using int.

So, to solve the “Big Sorting” problem just use Decimal instead of int:

from decimal import Decimal
numbers_array = ['9', '1', '3', '9'*998000]
sorted(list(map(lambda number: Decimal(number), numbers_array)))

The smarter implementation takes 0.07 seconds to run on my CPU.

Alembic Migrations: How Execute Raw SQL On New Tables

Problem

You have Alembic migrations. The current migration creates a new table and you want to insert rows into that table using raw SQL.

You are trying to establish a connection with the database using a DBAPI such as psycopg2, but when you try to insert rows you see an error that says something like the table doesn’t exist yet.

Your migration might look something like this:

import psycopg2
import sqlalchemy as sa
from alembic import op

def upgrade():
    op.create_table(
        'person',
        sa.Column('id', sa.Integer(), nullable=False),
        sa.Column('name', sa.String(), nullable=False),
    )

    bind = op.get_bind()
    session = Session(bind=bind)

    people = ['Harry', 'Ron', 'Hermione']

    conn = psycopg2.connect('postgres://username:password@localhost/dbname')
    cursor = conn.cursor()

    insert_person_sql = "INSERT INTO person (name) VALUES ('{name}');"
    for person in people:
        cursor.execute(insert_person_sql.format(name=person)

    conn.commit()

When attempting to run the migration there is a failure on the line cursor.execute(...).

Why does this happen? I believe the reason for this is that Alembic manages migrations as a transaction. The Alembic DB session created the person table, but doesn’t commit the change to the database until the entire migration is complete.

Therefore, the other connection, conn, established with the psycopg2 library is not aware of the person table while the migration is still running.

Solution

If you find yourself in this situation, the solution may be to bind to the Alembic session. Then you can execute SQL in a context that is aware of the new person table.

The new code will look something like this:

import sqlalchemy as sa
from alembic import op
from sqlalchemy.orm import sessionmaker

Session = sessionmaker()

def upgrade():
    op.create_table(
        'person',
        sa.Column('id', sa.Integer(), nullable=False),
        sa.Column('name', sa.String(), nullable=False),
    )

    bind = op.get_bind()
    session = Session(bind=bind)

    people = ['Harry', 'Ron', 'Hermione']

    insert_person_sql = "INSERT INTO person (name) VALUES ('{name}');"
    for person in people:
        session.execute(insert_person_sql.format(name=person)

Note, Alembic components such as the revision and down_revision field have been omitted for brevity, as well as the downgrade() function.

Gotchas of Opening CSV in Excel

Sometimes Excel converts CSV integers into scientific notation and/or truncates decimal points. This behavior is undesirable in some cases. There is an easy way to avoid this (see below). But before we apply the solution we need to know: when exactly does this happen?

Integers

10 digits is safe for integers, 11 digits is converted to scientific notation.

Inserting a tab character before an 11 digit integer prevents Excel from converting the number to scientific notation.

Here is some Python code that will insert the tab for appropriate cell values.

if len(cell_value) > 10:
  return "\t%s" % s

For example, the CSV input would be:

somevalue,12345123451,anothervalue,1234512345

And the CSV output would be:

somevalue,	12345123451,anothervalue,1234512345

Notice that a tab was inserted before the integer that was greater than 10 digits long, while the integer that was exactly 10 digits long was untouched.

Decimals

15-n decimal places are kept the rest are thrown out, where n is the number of digits left of the decimal point. For example, if 2 digits are to the left of the decimal point then 13 digits are kept to the right of it.

Inserting a tab character before a decimal with more than 15 total digits will make Excel keep all digits.

Wrapping Up

These two behaviors when opening CSV in Excel are consistent regardless of the column location of the value. In other words, it can be at the beginning, end or middle of a series of columns.

In each case, inserting a tab character will stop Excel from converting integers to scientific notation and truncating decimals.

These tests were performed with Microsoft Excel for Mac, version 16.9 (180116).

Python Arrays: Extend vs Append

It is important to know the difference between extend and append when working with arrays in Python.

Let’s say we have an example array: my_array = [].

If an array is passed to append, (i.e. my_array.append([1, 2]), then my_array will become a 2D array. It will look like this: [[1,2]]. In other words, append won’t flatten the arrays together into a 1D array.

However, it is possible to combine two arrays like this and end up with a 1D array using Python’s extend method. Calling my_array.extend([1, 2]) will result in my_array looking like this: [1,2].

Here is an example using the Python REPL:

>>> a = [1]
>>> b = [1, 2]
>>> c = [1, 2, 3]
>>> a.append(b)
>>> a
[1, [1, 2]]
>>> c.extend(b)
>>> c
[1, 2, 3, 1, 2]

Python 3.5 introduced another way to combine arrays. The unpacking operator * can be used in the following way:

>>> [1, 2, 3, *[4, 5, 6]]
[1, 2, 3, 4, 5, 6]

Note, the unpacking approach does not work in Python 2.7.14.

How-To Run a Specific Test Only from a PyTest Suite

Put @pytest.mark.<identifier> above the test method.

Run $ pytest -v -m <identifier>.

<indentifier> can be anything you choose.

For example, the a test file may look like this:

import pytest

@pytest.mark.focus
def first_test():
  # do a test here
  print('first test')

def second_test():
  # do another test here
  print('second test')

And then you can run the following command: pytest -v -m focus. first test should be printed to stdout, indicating only the first test was run.

Don’t forget to import pytest.

The -v flag will make the output more verbose so you can see everything that is going on.

The -m flag expects one argument: a mark expression. It will tell pytest to only run tests matching the identifier given.