# Git Deep Dive

A Version 1 Technical Meetup talk covering the deep internals of git.


## Introduction

This is the slide-deck and set-up scripts used to give the *Git Deep Dive* technical meet-up talk on 2018-06-25 in Version 1, by Éibhear Ó hAnluain.

The pack comes with the following:

-   This document
-   The slide-deck, `GitDeepDive.pdf`
-   A script to set up a simple git repository for exploration purposes: `simple-setup.sh`
-   A script to set up the mimic of a development team and processes: `gitDemo.sh`
-   A script to set up another simple repository to run through the process for completely clearing a file out of a git database: `largeFile-setup.sh`.
-   A `.gitignore` file.


## Setup

1.  Run the script `simple-setup.sh`. This will create a git repository in `simpleRepo` and populate it with some commits. The commits in this repository will correspond to the diagrams in the slide-deck that relate to merging.
2.  Run the script `largeFile-setup.sh`. This will create another git repository in `largeFileRepo` and populate it with the same commits as in `simpleRepo`, and then puts in some more to create and manipulate a large binary file, and then to remove it. This will be used for the demonstration to remove a file completely from the repository.
3.  Run the script `gitDemo.sh`. This will replicate a previous git repository as a bare repo in `devTeamDemo/javaBootcampNoEclipse.git`, and then create a number of clones to represent the actions of a team lead and two developers, and mimic collaborative development among them. `gitDemo.sh` takes one optional parameter, `-s`, which will cause the progression to stop for 15 seconds following each "actor's" push in order to afford the opportunity to look at the git log.
4.  Use the `gpg` utility to generate a key-pair so that you can work through the commit- and tag-signing slide and sample commands. If you have access to `gpg2` and not `gpg`, you'll need to set the following `git config` to make sure it's picked up: `git config --global gpg.program gpg2`.
5.  Open the file `GitDeepDive.pdf`, and as you go through the slides, refer to the *Sample Commands* section below for commands you can execute to get further insights.


## Sample commands


### Slide: Configuration

    # Go to where the GitDeepDive.pdf file is and set your BASE_DIR
    # environment variable.
    cd <where-you-have-this-repo-cloned-to>
    export BASE_DIR=$(pwd)
    # List your basic git config
    git config --list
    # Go into the simple repo location and look at the various config
    # contexts
    cd ${BASE_DIR}/simpleRepo
    git config --local --list
    git config --global --list
    git config --system --list
    # Set some settings
    git config --global user.name "<YourName>"
    git config --global user.email "<YourEmailAddress>"
    # Use a text editor to edit your config
    git config -e


### Slide: fetch and merge, not pull

    # Go into the clone belonging to one of the developers in the
    # development team demo area
    cd ${BASE_DIR}/devTeamDemo/javaBootcampNoEclipse.dev1
    # Update the clone, but don't merge anything
    git fetch --prune
    # Review the local and remote branches.
    git branch -va


### Slide: Merging approaches: fast-forward

    cd ${BASE_DIR}/simpleRepo
    # Check out the master branch and review it's log
    git checkout master
    git log --decorate --graph --oneline --all
    # Merge in the Rel1 branch and review the new log.
    git merge Rel1
    git log --decorate --graph --oneline --all


### Slide: Merging approaches: merging strategies

    cd ${BASE_DIR}/simpleRepo
    # Check out the master branch, merge the Rel2 branch and review the
    # new log.
    git checkout master
    # You'll be prompted for a commit message here.
    git merge Rel2
    git log --decorate --graph --oneline --all


### Slide: Merging approaches: Rebase

    # Remove and refresh the simpleRepo
    ${BASE_DIR}/simple-setup.sh
    cd ${BASE_DIR}/simpleRepo
    
    # Check out master, review its log, merge Rel1 and review the log
    git checkout master
    git log --decorate --graph --oneline --all
    git merge Rel1
    git log --decorate --graph --oneline --all
    
    # Check out Rel2
    git checkout Rel2
    # Rebase Rel2 onto the now-new master
    git rebase master
    # Review the log
    git log --decorate --graph --oneline --all


### Slide: refs

    # Remove and refresh the simpleRepo
    ${BASE_DIR}/simple-setup.sh
    cd ${BASE_DIR}/simpleRepo
    
    # Look at the contents of .git/refs/heads and one of the heads itself
    ls -l .git/refs/heads/
    cat .git/refs/heads/Rel2
    
    # Tag a branch, look at the branch ref and the tag ref
    git tag Rel1.0 Rel1
    cat .git/refs/heads/Rel1
    cat .git/refs/tags/Rel1.0
    
    # Look at the HEAD ref
    cat .git/HEAD
    
    # Go to another clone and look at the refs for the origin remote
    cd ${BASE_DIR}/devTeamDemo/javaBootcampNoEclipse.dev1
    ls -l .git/refs/remotes/origin/
    # .git/packed-refs contains the refs that haven't been interacted with
    # yet.
    cat .git/packed-refs


### Slide: Annotated tags

    cd ${BASE_DIR}/simpleRepo
    
    # Look at a "lightweight" tag
    cat .git/refs/tags/Rel1.0
    # What type of object is it pointing to?
    git cat-file -t $(cat .git/refs/tags/Rel1.0)
    # What's in the object it's pointing to
    git cat-file -p $(cat .git/refs/tags/Rel1.0)
    
    # Create an annotated tag and look at *it*
    git tag -a -m "Formal release of 1.0" Rel1.0.prod Rel1
    cat .git/refs/tags/Rel1.0.prod
    # What type of object is it pointing to?
    git cat-file -t $(cat .git/refs/tags/Rel1.0.prod)
    # What's in the object it's pointing to
    git cat-file -p $(cat .git/refs/tags/Rel1.0.prod)


### Slide: blame

    # Look at the lines of the file on master
    git checkout master
    git blame information.md
    # .. and on Rel1
    git checkout Rel1
    git blame information.md
    # And on master after Rel2 has been merged in.
    git merge Rel2
    git blame information.md
    # Slide


### Slide: Tag and commit signing

    # Look at your secret keys
    gpg --list-secret-keys
    
    # Check out master and make a change to information.md
    git checkout master
    cat <<EOF >> information.md
    An additional line for demonstrating commit-signing.
    
    EOF
    
    # Add and commit the change, signing the commit.
    git add information.md
    git commit -S<secretKeyID> -m "Update to information.md"
    
    # Look at the commit object
    git cat-file -p master
    
    # Create an annotated tag from the Rel2 branch, signing it.
    git tag -s -u <secretKeyID> -m "Release 2." Rel2.0 Rel2
    # Look at the tag object
    git cat-file -p Rel2.0
    
    # Verify the signed tag and the signed commit.
    git tag -v Rel2.0
    git log --show-signature -1


### Slide: Git Objects &#x2013; blobs

    # Generate the SHA1 of the contents of the ~/.bash_history file as
    # though git would
    cat ~/.bash_history | git hash-object --stdin


### Slide: Git Objects &#x2013; trees

    cd ${BASE_DIR}/devTeamDemo/javaBootcampNoEclipse.dev1
    # Get the tree object for the latest version of the top-level of the
    # project
    git cat-file -p master | grep '^tree'
    # Look at the contents of that tree object:
    git cat-file -p $(git cat-file -p master | grep '^tree' | sed 's/^tree //')


### Slide: Git Objects &#x2013; commits

    cd ${BASE_DIR}/simpleRepo
    git log --graph --all --decorate --oneline
    git cat-file -p HEAD
    git cat-file -p <anyOtherCommitId>


### Slide: Git Objects &#x2013; tags

    # Find all the tag objects
    git cat-file --batch-check='%(objecttype) %(objectname) %(objectsize)' --batch-all-objects | grep tag
    git cat-file -p <anyOfTheTagObjects>


### Slide: "Content Addressable Filesystem"

    # Find all the objects, and select one that refers to a file
    git rev-list --all --objects
    # Look at the contents of the selected object
    git cat-file -p <selectedBlobObject>
    # Use git to get the SHA1 of the contents of that object
    git cat-file -p <selectedBlobObject> | git hash-object --stdin
    # The name of the file for that object is based on the SHA1.
    ls -l .git/objects/...


### Slide: The reflog

    # Look at the reflog, then clear it completely and look at it again.
    git reflog
    git reflog expire --expire=now --expire-unreachable=now --verbose --all
    git reflog


### Slide: fsck and gc

    git fsck
    git gc


### Slide: Useful commands

    # Go into the repo where the large file had been created
    cd ${BASE_DIR}/largeFileRepo
    # List all the objects in increasing order of object size.
    git cat-file --batch-check='%(objecttype) %(objectname) %(objectsize)' --batch-all-objects | sed -n 's/^blob //p' | sort -n --key=2
    # Look for the file name associated with an object
    git rev-list --objects --all | grep <blobID>
    # Look for the commits that made changes to a specific file
    git log --follow -- "largeInformation.md"


### Slide: Permanently removing a file from your git db

    
    # Preserve information on the tags, as you may need this later.
    for tag in $(git tag)
    do
      echo "${tag},$(git log --format="%H,\"%cn\",\"%ci\",\"%s\"" ${tag} | head -1)"
    done | tee /tmp/tag_list.csv
    
    # Create a local branch for all the remote branches. This does nothing
    # in this demo as there is no remote.
    for branch in $(git branch -r | grep -v HEAD | sed 's/\ \ origin\///')
    do
      git branch ${branch} origin/${branch}
    done
    
    # Check out each branch and determine the amount of space it uses
    for branch in $(git branch | sed 's/^..//')
    do
      git checkout -q ${branch}
      du -sk . | sed "s/\./${branch}/"
    done | tee /tmp/branch_sizes.out
    
    # Check out the Rel1 branch
    git checkout Rel1
    # Use git-filter-branch to remove the file from all the commits on the
    # branch
    git filter-branch --tree-filter 'rm -f largeInformation.md' --prune-empty HEAD
    # Update refs (will fail for some branches)
    git for-each-ref --format="%(refname)" refs/original/ | xargs -n 1 git update-ref -d
    
    # Do the same for all the other branches
    for branch in Rel2 Rel3 Rel4 master
    do
      git checkout ${branch}
      git filter-branch --tree-filter 'rm -f largeInformation.md' --prune-empty HEAD
      git for-each-ref --format="%(refname)" refs/original/ | xargs -n 1 git update-ref -d
    done | tee /tmp/cleanup.out
    
    # Clear out the reflog
    git reflog expire --expire-unreachable=now --all
    # Run the garbage collector on the repository
    git gc --prune=now
    # Run an FSCK on the repo
    git fsck --unreachable --no-reflogs
    
    # Check out each branch again and determine the amount of space it
    # uses
    for branch in $(git branch | sed 's/^..//')
    do
      git checkout -q ${branch}
      du -sk . | sed "s/\./${branch}/"
    done | tee /tmp/branch_sizes_post_process.out