Skip to content

Author: Bobby Lin

Walkthrough – Solidity Smart Lottery project

Published by Bobby Lin on March 6, 2022

The source of the project is from:

https://github.com/PatrickAlphaC/smartcontract-lottery
https://www.youtube.com/watch?v=M576WGiDBdQ&t=22298s

The purpose of this walkthrough is to explain each part of the Smart Contract code with more context so that we can gain better understanding.

Lottery.sol

Think about the different processes for a lottery.

  1. Participants have to enter the lottery. We need to track down who are the participants for this lottery.
  2. We need to know how much to charge the participants for entering the lottery.
  3. Owner will need to start the lottery.
  4. Owner will end the lottery and we need a random way to calculate the winner of the lottery. And we need a robust way of generating the random winner that minimise the probability of participants cheating.
  5. We need to track the state of the lottery: whether it is opened, closed or the winner is being calculated.

Tracking Participants

We need to create an address array to track down the participants. Before we add the new player to the participant list, they have to pay an entry fee. This entry fee is calculated based on current Ether price.

address payable[] public players;

function enter() public payable {
    require(msg.value > getEntranceFee());
    players.push(msg.sender);
}

We can use ChainLink price feed to get the latest Ether data. To do so, we need to specify the dependencies and remappings in brownie-config.yaml file.

import "@chainlink/contracts/src/v0.6/interfaces/AggregatorV3Interface.sol";

[...]
AggregatorV3Interface internal ethUsdPriceFeed;

function getEntranceFee() public view returns (uint256) {
    (, int256 price, , , ) = ethUsdPriceFeed.latestRoundData();
    uint256 adjustedPrice = uint256(price) * 10**10;
    uint256 costToEnter = (usdEntryFee * 10**18) / adjustedPrice;
    return costToEnter;
}

Walkthrough – Solidity Local Development setup (with Python)

Published by Bobby Lin on February 26, 2022

Requirements

  • Install python3 and the following python packages:
pip install web3 py-solc-x
  • VS code IDE and extensions: Solidity (Juan Blanco), Python (Microsoft)

We are going to walkthrough a tutorial created by Patrick Collins (https://github.com/PatrickAlphaC/web3_py_simple_storage)

  • Install nodejs
  • Install ganache using yarn or npm (to simulate a local blockchain VM)

Setting and Deploying the Smart Contract

Create a new project folder and copy the sample solidity code to a new file SimpleStorage.sol:

// SPDX-License-Identifier: MIT

pragma solidity >=0.6.0 <0.9.0;

contract SimpleStorage {
    uint256 favoriteNumber;

    // This is a comment!
    struct People {
        uint256 favoriteNumber;
        string name;
    }

    People[] public people;
    mapping(string => uint256) public nameToFavoriteNumber;

    function store(uint256 _favoriteNumber) public {
        favoriteNumber = _favoriteNumber;
    }

    function retrieve() public view returns (uint256) {
        return favoriteNumber;
    }

    function addPerson(string memory _name, uint256 _favoriteNumber) public {
        people.push(People(_favoriteNumber, _name));
        nameToFavoriteNumber[_name] = _favoriteNumber;
    }
}

Create a deploy.py file. We are going to walkthrough step-by-step to understand each part of the code.

1. Import these required modules. 

from solcx import compile_standard
from solcx import install_solc
import json
import os
from web3 import Web3

2. We use solcx to compile the smart contract code. This is a python wrapper for the Solidity compiler.

First we open and read the content of the smart contract code.

Then install the solidity compiler version. It will download the compiler from the project’s Github download link.

Configure the compile standards:

  • language
  • sources
  • settings
  • solidity compiler version

We can dump the compiled code to see the structure of the code.

with open("./SimpleStorage.sol", "r") as file:
    simple_storage_file = file.read()

print("Installing solc...")
install_solc('0.6.0')

# Configure compile standards
compiled_sol = compile_standard(
    {
        "language": "Solidity",
        "sources": {"SimpleStorage.sol": {"content": simple_storage_file}},
        "settings": {
            "outputSelection": {
                "*": {
                    "*": ["abi", "metadata", "evm.bytecode", "evm.bytecode.sourceMap"]
                }
            }
        }
    },
    solc_version="0.6.0"
)

# Dump the compiled code to see the structure of the code
with open("compiled_code.json", "w") as file:
    json.dump(compiled_sol, file)

3. Get the bytecode and abi from the compiled code. How do we know the structure of the JSON? Refer to the compiled_code.json file

bytecode = compiled_sol["contracts"]["SimpleStorage.sol"]["SimpleStorage"]["evm"]["bytecode"]["object"]

abi = compiled_sol["contracts"]["SimpleStorage.sol"]["SimpleStorage"]["abi"]

4. Now start ganache server and we can see the generated dummy accounts and private keys. Note down where the server is listening. 

$ ganache
ganache v7.0.2 (@ganache/cli: 0.1.3, @ganache/core: 0.1.3)
Starting RPC server

Available Accounts
==================
(0) 0xB136383615B477B1B816f4227A509ea8F0C0c9DD (1000 ETH)
(1) 0xB6c6BDb34A834BAcc8e07c9765E2f85D1619beDc (1000 ETH)
(2) 0xb2e141ed4EF4F30BC7a1848FFbd623b19B08608C (1000 ETH)
(3) 0x32c0DB620E7355feE0254813932a4E7a454D74f8 (1000 ETH)
(4) 0xB19D507aEE3BdA3c9da9b360E491B88FFd857f14 (1000 ETH)
(5) 0x7B0929a005B39Cce2C9795558371F3865Cff1Bf9 (1000 ETH)
(6) 0x13665EC9cEE2915402BD7Ce69c05F70E9CBCF2ef (1000 ETH)
(7) 0x2C2c3C4585c9425969C03055554dD0C15f5a57b8 (1000 ETH)
(8) 0xdAD8Ae2871Cb242C001A68EB5Bc6941BFDB0d2A7 (1000 ETH)
(9) 0x85214561dCD632581a0b60BeE5989607005BE663 (1000 ETH)

Private Keys
==================
(0) 0xa4c6bac88b45ba1e21eafbd736c92ca60b67bbfb956ccd3da37fa6f83ebe38c1
(1) 0xfa197f239d6df371b0242b8fe96b0d1883a392ff5ab4502cefae0e972f07f081
(2) 0x6c96d2d7b0fb9b56cad49887e3f198de2faa93d321240201d0572732f83bbcc8
(3) 0xd9eb1ec171c1aee37b0603b85d58b615d678f3f9c85c4e5fe31c322113d009d3
(4) 0xd6c212925da4e19a4708b43d1728efef1cf5c839fd44ee411107837b30d8e38c
(5) 0x048c50d14357791a5cbf1ecbf1febda6bf215f946bac50fac410524cde6cb397
(6) 0xb9eeb83abac9b23a5bc801e584c646f92ecb37cd4b7770100f84ea2ccdf3a304
(7) 0xad157fbf68f3e7fb3047ca653c073b4389f5020ce8d397c7c8f3533c491a15ba
(8) 0x01caede47d8ad5d0e5c125b9d4ceba7abd201a26be10730fc926f673f275fd42
(9) 0x07da7e22ff6a56ce2c07bf724913e507621181f369d481fd94ebb12d577d4650

HD Wallet
==================
Mnemonic:      team shoot anchor limit inform imitate melody decrease wing sadness orange mammal
Base HD Path:  m/44'/60'/0'/0/{account_index}

Default Gas Price
==================
2000000000

BlockGas Limit
==================
30000000

Call Gas Limit
==================
50000000

Chain Id
==================
1337

RPC Listening on 127.0.0.1:8545

5. Setup connection to the ganache server


w3 = Web3(Web3.HTTPProvider("http://127.0.0.1:8545"))
chain_id = 1337
my_addr = "0xB136383615B477B1B816f4227A509ea8F0C0c9DD"
private_key = os.getenv("PRIVATE_KEY")

DO NOT HARDCODE YOUR PRIVATE KEY IN YOUR CODE REPOS

If we want to connect to Testnet environment, we can use an ETH gateway such as Infura or Alchemy.

6a. Create the contract with the provider using the abi and bytecode.

SimpleStorage = w3.eth.contract(abi=abi, bytecode=bytecode)

6b. Get latest transaction count and use it as the nonce

nonce = w3.eth.getTransactionCount(my_addr)

6c. Submit the transaction that deploys contract using the chainId, gasPrice, from (which addr) and nonce

transaction = SimpleStorage.constructor().buildTransaction(
    {
        "chainId": chain_id,
        "gasPrice": w3.eth.gas_price,
        "from": my_addr,
        "nonce": nonce
    }
)

6d. Sign the transaction with private key

signed_txn = w3.eth.account.sign_transaction(transaction, private_key=private_key)

7. Deploy the contract

tx_hash = w3.eth.send_raw_transaction(signed_txn.rawTransaction)tx_receipt = w3.eth.wait_for_transaction_receipt(tx_hash)contract_addr = tx_receipt.contractAddressprint(f"Contract is deployed to {contract_addr}")

Using the Smart Contract

Since the Smart Contract is deployed, we can provide the contract address and abi to create the smart contract (“simple_storage”).

simple_storage = w3.eth.contract(address=contract_addr, abi=abi)

We can make a function call without changing any state in the smart contract. In this example, we are trying to retrieve the stored value in the smart contract.

print(f"Initial Stored Value = {simple_storage.functions.retrieve().call()}")

Now, we call the store function in the smart contract to update favoriteNumber variable.

greeting_transaction = simple_storage.functions.store(15).buildTransaction(
    {
        "chainId": chain_id,
        "gasPrice": w3.eth.gas_price,
        "from": my_addr,
        "nonce": nonce + 1
    }
)

We will sign this transaction with the private key, send the transaction to the Ganache server and then wait for the transaction receipt.

Notice if you execute the transaction in local blockchain VM, the transaction speed will be very fast. But in actual Testnet or Mainnet, the transaction is likely to be slower.


signed_greeting_txn = w3.eth.account.sign_transaction(greeting_transaction, private_key=private_key)

tx_greeting_hash = w3.eth.send_raw_transaction(signed_greeting_txn.rawTransaction)

tx_receipt = w3.eth.wait_for_transaction_receipt(tx_greeting_hash)

Let’s print the stored value and we can see it is changed to 15

print(simple_storage.functions.retrieve().call())

How to Dockerize your project?

Published by Bobby Lin on January 30, 2022

Creating a Dockerfile

Usual steps of creating a Dockerfile of your application:

  1. Specify Base Image
  2. Specify any environment variables
  3. Specify commands to run on the top of OS layer such as:
    • downloading or updating dependencies
    • add groups or users to run process instead of root
  4. Copy the application files into the image.
  5. Specify which port(s) for container to listen
  6. Specify the command to start the application
Dockerfile InstructionsArguments
FROMBase Image or other Container Image
RUN1. Passing commands (runs in shell)
2. Passing exec form: [“executable”, “p1”, “p2”]
Execute command in a new layer.

Examples:
– Updating dependencies
COPYCOPY is preferred to ADD as we know clearly that COPY will just copy files from local directory. ADD is useful if you wanna copy the content in a tar file.
ENTRYPOINTDefine the command that will always be executed when the container starts.

By default, Docker is using  /bin/sh -c as the entrypoint.
CMDDefine the arguments that will be appended to the ENTRYPOINT
WORKDIRDefine where your commands should run. Saves the trouble of running to many cd .....
EXPOSEDefine the ports that a container listens.
ENVSetting environment variables that can be used in other instructions such as RUN
FROM node:12-alpine
RUN apk add --no-cache python2 g++ make
EXPOSE 9091
WORKDIR /app
COPY . .
RUN yarn install --production
CMD ["node", "src/index.js"]

References:

Running a container

Make sure the port publish setting (-p) is positioned before the image tag so that it can override the image default settings.

https://stackoverflow.com/questions/66316400/cannot-reach-docker-container-port-not-bound
docker run -p 9091:9091 -t spring-boot-docker

References:

Purpose of DevSecOps and its future

Published by Bobby Lin on January 8, 2022

“…make sure that the constraint is not allowed to waste any time. Ever. It should never be waiting on other resource for anything, and it should always be working on the highest priority commitment the IT Operations organization has made to the rest of the enterprise. Always.”

The Phoenix project

The NUMBER ONE constraint in Security department is people.

It is unlikely we can hire enough people to match the number of developers and operations engineers.

The way to free up our constraint (people) is to try to automate as many tasks as possible so that the people can do the things that are unique and contextual.

Another way is to take a preventative approach by educating developers and ops engineers on best security practices that they need to follow (this means secure by defaults configurations, having documentation and guides). The famous Netflix’s paved roads….

any improvement not made at the constraint is just an illusion, yes?

The Phoenix project

The Programmer’s Brain – Improving your skills in reading and writing code

Published by Bobby Lin on November 14, 2021
https://www.amazon.com/Programmers-Brain-every-programmer-cognition/dp/1617298670

Performing source code review is one of the important skills that you should pickup as an Application Security engineer. Being polygot in programming is helpful because you might be reviewing source code that are written in different languages. Right now, I seldom see anyone mentioning a systematic approach to read source code if you are a novice with the codebase or languages.

Reading source code is a underrated skill in today’s programming education. Often when we want to learn programming, we are given advice to build projects and write more code to learn programming. However, another aspects of learning programming is to read the code of other programmers.

This is why I think “The Programmer’s Brain” is one of the insightful programming books to read because it discusses about different aspects of being a programmer:
– How to get better at reading code?
– How to get better at thinking about code?
– How to get better at writing code?

Get better at reading code!

Cognitive Processes in relations to programming

We can model our cognitive processes with the below diagram:

Hermans, F. (2021). Programmer’s brain: What every programmer needs to know about cognition. Manning.

At a high level, when we start reading the code, information relating to the code enters to our Short-term memory. Think of the time when you remember some information for a short period of time to memorize a phone number or a quick task to complete. This is the use of short-term memory.

Then when we are thinking and interpreting the code, we are using the information from our Short-term memory to our Working memory. The information is processed in our Working memory to generate new understanding / meaning.

Our Working memory retrieves and connect information from our Long-term memory to process the information. Think of our Working memory like a melting pot. For example, we can recall certain syntax pattern from our Long-term memory. Like when we are reading the Javascript code, we know console.log(...) will print out information based on our Long-term memory.

console.log(message)

In short, when we are reading a code, different cognitive processes are engaged to comprehend the code and perform certain actions later such as changing the code or adding new code etc.

Why we get confused when reading code?

1) Lack of knowledge

Sometimes you get confused when reading a source code if you are unfamiliar with the language syntax or concepts. Or you might be unfamiliar with the specific industry / domain knowledge that the code is written for. In this context, we say that you lack knowledge to understand the code.

For example, below is a snippet of a Racket code. If you do not understand LISP-like code or concept of Lambda, then you will not know how to evaluate ((double inc) 5) 

(define (inc n) (+ n 1))
(define (double f)
  (lambda (x) (f (f x))))

((double inc) 5)

To understand what is going on:
– You need to know the syntax of a function in LISP
– You understand that you can pass function into another function
– In this case, the function inc is passed to the function double. In this function double, the inc function will be applied twice.
– Hence ((double inc) 5) will evaluate to 7.

Example code here

To understand in terms of cognitive processes, we are unable to retrieve any information from our Long-term memory that can be used by the working memory to evaluate the code.

2) Lack of information

Unfamiliar with how a particular method works or purpose of a class etc because the information cannot be retrieved directly from the code itself.

For example, we can see a python function that seems to filter a group of members by name. But we do not know how exactly the name is gonna be filtered. Hence we might guess or search for additional information from the code base.

We are temporarily confused because we cannot understand the function immediately from the code itself.

def filter_members(all_members):
  return filter_by_name(all_members)

3) Lack of processing power

Too many items to hold in your working memory.

Code is written in too complex manner.

Why reading unfamiliar code is hard?

Types of cognitive loads

Intrinsic

Extraneous

Germane

Difference between Experts and Novice?

Clunking

Learning language syntax quickly

While coding a program, we might forget about how a particular concept works or their syntax. Often, we will just quickly google to retrieve for an answer (either from the documentation or stackoverflow).

The author argues it is better to know some of these syntax by heart rather than googling for answers all the time. First of all, it is distracting to google for the answers as you might be tempted to do something else (once you are in your browser especially with the multi-tabs). Second, you need to be able to recall the language syntax from your long-term memory in order to clunk the code (that you are reading) effectively.

Our long-term memories will decay in a pattern similar to a forgetting curve. If we don’t recall the things that we learned in a specific period of time, we will forget things. But if we try to recall the memories at a specific time, the decay will be slowed down.

We know this very well when we tried to cram for an exam, then we might forget everything a few things after the exam is over. Because of the failure of having future reminders, we will struggle a retrieve a concept or syntax from our long-term memory.

Want to Remember Everything You'll Ever Learn? Surrender to This Algorithm | WIRED
https://www.wired.com/wp-content/uploads/archive/images/article/magazine/1605/ff_wozniak_graph_f.jpg


What can you do to recall the syntax effectively?

It is not efficient to recall every concepts every single day. Instead, we can use a spaced repetition system (SRS) software to automate the reminders for us based on how well we think we can recall a particular concept.

Note: I have tried spaced repetition system and struggles to integrate to my everyday workflow. It’s not because SRS methodology does not work. Rather it takes a commitment to adopt SRS well and make it a habit go through your deck everyday. I will keep trying and see what are the ways we can integrate SRS better into our lives.

Adopt a spaced repetition system (SRS) and add new knowledge that you have to your deck. This can be a situation where you are learning a new programming language, a new concept or framework. You will need to use judgments to know what concepts or syntax need to be included as you can’t add everything you learned into the deck.

Adding a new card on Javascript’s array prototype filter method

Also when you find yourself googling for an answer, then add a new card to your deck. This shows that you have not understand the concepts or syntax by heart. For example, most programmers would know how to write a for-loop in their language. If they do not know, it means that they have not learned the language deeply yet.

Further reading on SRS

Memorizing a programming language using spaced repetition software by Derek Sivers

Augmenting Long-term Memory

Spaced Repetition for Efficient Learning by Gwern

Effective learning: Twenty rules of formulating knowledge

Another way to recall the syntax better is to actively think about the concepts that you are studying. It is easier for us to recall something if they are related to something that we know. When we relate a new concept / syntax to our existing knowledge, then we have a better chance of recalling the new concept / syntax in the future. Some questions to think about can be:

  • Think and write down the concepts that you think is related to this new concept or syntax.
    • In what ways are they similar and different?
  • Think of variants of code that can achieve the same goals as this new concept or syntax.
  • How important is this concept or syntax to the language, framework or codebase etc.?

Reducing cognitive loads when reading complex code

  • Refactoring code temporarily
  • Replacing unfamiliar language constructs
  • Adding the concepts that you are confused to SRS.
  • Working memory aids
    • Create Dependency graph
    • Using a state table

Think about code better!

Reaching deeper understanding of the codebase

Get better at solving programming problems

Avoiding bugs (misconceptions in thinking)

Write better code!

Naming things better

  • Name moulds
  • Feitelson’s three-step model

Avoiding code smells and cognitive loads

  • 22 code smells from Martin Fowler’s Refactoring
  • Arnaoudova’s six linguistic antipatterns

Get better at solving complex programming problem

  • Automatization
  • Learn from code and its explanation
  • Germane Load

Practices that you can do


A) Reading different code base and attempt to understand what each code is doing.

1. Choose a code base to read

Choose a codebase where you have at least some knowledge of the programming language. You should have a high level understanding of what the code does.

2. Select a code snippet and study it for two minutes.

Choose a method, function or coherent code that is about half a page or maximum 50 lines of code.

3. Reproduce the code in paper or in an IDE (new file).

4. Reflect on what you have produced.

Which lines do you find easy and which lines are difficult?

Does the lines of code that are unfamiliar to you because of the programming concepts or domain knowledge?

B) Reading and using more programming concepts

Write-up on Capture the Ether

Published by Bobby Lin on October 18, 2021

Warmup:

Notice that you will be paying gas fees for each contract that you try to deploy
  • Call me
    • https://capturetheether.com/challenges/warmup/call-me/
    • Notice how slow are Ethereum transactions 😃
    • Note: Metamask does not auto-inject web3.js into the browser (info: https://docs.metamask.io/guide/provider-migration.html#summary-of-breaking-changes)
    • Also view this write-up for more details: https://medium.com/coinmonks/lets-play-capture-the-ether-warmup-68a5fc38e670
    • Use Remix IDE to call the contract:

Using ACTA methodology to Bug hunting writeups

Published by Bobby Lin on September 26, 2021

Ideas

In the security industry, we are lucky to have many hunters sharing their write-ups in public. To learn deeper from these write-ups, we should have some methodologies to extract knowledge from these experts.

Applied Cognitive Task Analysis (ACTA) is a methodology used by researchers to elicit knowledge from different domain experts. I think it is an interesting methodology to experiment with to see if we can extract knowledge in a structured way from the bug hunting experts.

Note: I will be experimenting with a few techniques from ACTA and adjusting it to secondary texts like bug write-ups, blogs, youtube videos, news, Github commits etc. instead of the interviews with experts.

The goal is to see the contrast between Novice bug hunters like me and Experts bug hunters and then use the learned knowledge to improve my own hunting methodology.

Notes on Learning from Huberman

Published by Bobby Lin on August 1, 2021
https://unsplash.com/photos/3b2tADGAWnU

How do we trigger neuroplasticity?

  • High Focus
  • High Alertness

Self-assessment

  • Monitor your alertness rhythm (Do you tend to be more alert in the morning or evening?)
  • When trying a productivity technique, you need to evaluate if it increases your autonomic arousal level.

High Alert state

  • You do not need music for focusing
  • Good for performing linear implementation of work (where you already know what you are going to do)

Low Alert state

  • Some music can help with increasing alertness
  • Good for performing some creative brainstorming and implementation of creative ideas
  • Deep Rest for 30 mins in the late afternoon (when you are feeling less alert)
    • https://www.youtube.com/watch?v=iEw5BkK9K9A&t=15s

Useful Heuristics

  • Get sunlight during the first 30 mins of the day
  • Delay caffeine intake for 2 hours after waking
  • Hydrates yourself after waking up and throughout the day
  • Exercise early in the day can help with High alertness and activate the tendency for action.
  • Avoid light exposure between 10pm to 4 am (use Dim light).

Diet

  • Fasting in the morning.
  • First meal at middle of day
    • Eat as little carbs as possible as it makes us feel sleepy
    • More protein and salad
  • Evening meal
    • More carbs (as it helps with sleep later)
    • Some protein

THM – Overpass 2

Published by Bobby Lin on July 4, 2021

Gaining Foothold

ssh james@[TARGET IP] -p 2222
Password: november16

Privilege Escalation

Option 1: Sudo

I tried to sudo but the credentials for James doesn’t work.

Password: whenevernoteartinstant

Previously I have cracked the password for these users but the credentials does not work as well

james:$6$7GS5e.yv$HqIH5MthpGWpczr3MnwDHlED8gbVSHt7ma8yxzBM8LuBReDV5e1Pu/VuRskugt1Ckul/SKGX.5PyMpzAYo3Cg/:18464:0:99999:7:::
paradox:$6$oRXQu43X$WaAj3Z/4sEPV1mJdHsyJkIZm1rjjnNxrY5c8GElJIjG7u36xSgMGwKA2woDIFudtyqY37YCyukiHJPhi4IU7H0:18464:0:99999:7:::
szymex:$6$B.EnuXiO$f/u00HosZIO3UQCEJplazoQtH8WJjSX/ooBjwmYfEOTcqCAlMjeFIgYWqR5Aj2vsfRyf6x1wXxKitcPUjcXlX/:18464:0:99999:7:::
bee:$6$.SqHrp6z$B4rWPi0Hkj0gbQMFujz1KHVs9VrSFu7AU9CxWrZV7GzH05tYPL1xRzUJlFHbyp0K9TAeY1M6niFseB9VLBWSo0:18464:0:99999:7:::
muirland:$6$SWybS8o2$9diveQinxy8PJQnGQQWbTNKeb2AiSp.i8KznuAjYbqI3q04Rf5hjHPer3weiC.2MrOj2o1Sw/fd2cu0kC6dUP.:18464:0:99999:7:::

Option 2: SUID

I tried to find the SUID binaries. There is one uncommon binary in /home/james/.suid_bash When I execute the binary, there is a bash running but the user was still james. But we can elevate the privilege but running with -p

That’s how we get the root user.