GitDeepDive/README.md

320 lines
11 KiB
Markdown

# Git Deep Dive
A Version 1 Technical Meetup talk covering the deep internals of git.
## Introduction
This is the slide-deck and set-up scripts used to give the *Git Deep Dive* technical meet-up talk on 2018-06-25 in Version 1, by Éibhear Ó hAnluain.
The pack comes with the following:
- This document
- The slide-deck, `GitDeepDive.pdf`
- A script to set up a simple git repository for exploration purposes: `simple-setup.sh`
- A script to set up the mimic of a development team and processes: `gitDemo.sh`
- A script to set up another simple repository to run through the process for completely clearing a file out of a git database: `largeFile-setup.sh`.
- A `.gitignore` file.
## Setup
1. Run the script `simple-setup.sh`. This will create a git repository in `simpleRepo` and populate it with some commits. The commits in this repository will correspond to the diagrams in the slide-deck that relate to merging.
2. Run the script `largeFile-setup.sh`. This will create another git repository in `largeFileRepo` and populate it with the same commits as in `simpleRepo`, and then puts in some more to create and manipulate a large binary file, and then to remove it. This will be used for the demonstration to remove a file completely from the repository.
3. Run the script `gitDemo.sh`. This will replicate a previous git repository as a bare repo in `devTeamDemo/javaBootcampNoEclipse.git`, and then create a number of clones to represent the actions of a team lead and two developers, and mimic collaborative development among them. `gitDemo.sh` takes one optional parameter, `-s`, which will cause the progression to stop for 15 seconds following each "actor's" push in order to afford the opportunity to look at the git log.
4. Use the `gpg` utility to generate a key-pair so that you can work through the commit- and tag-signing slide and sample commands. If you have access to `gpg2` and not `gpg`, you'll need to set the following `git config` to make sure it's picked up: `git config --global gpg.program gpg2`.
5. Open the file `GitDeepDive.pdf`, and as you go through the slides, refer to the *Sample Commands* section below for commands you can execute to get further insights.
## Sample commands
### Slide: Configuration
# Go to where the GitDeepDive.pdf file is and set your BASE_DIR
# environment variable.
cd <where-you-have-this-repo-cloned-to>
export BASE_DIR=$(pwd)
# List your basic git config
git config --list
# Go into the simple repo location and look at the various config
# contexts
cd ${BASE_DIR}/simpleRepo
git config --local --list
git config --global --list
git config --system --list
# Set some settings
git config --global user.name "<YourName>"
git config --global user.email "<YourEmailAddress>"
# Use a text editor to edit your config
git config -e
### Slide: fetch and merge, not pull
# Go into the clone belonging to one of the developers in the
# development team demo area
cd ${BASE_DIR}/devTeamDemo/javaBootcampNoEclipse.dev1
# Update the clone, but don't merge anything
git fetch --prune
# Review the local and remote branches.
git branch -va
### Slide: Merging approaches: fast-forward
cd ${BASE_DIR}/simpleRepo
# Check out the master branch and review it's log
git checkout master
git log --decorate --graph --oneline --all
# Merge in the Rel1 branch and review the new log.
git merge Rel1
git log --decorate --graph --oneline --all
### Slide: Merging approaches: merging strategies
cd ${BASE_DIR}/simpleRepo
# Check out the master branch, merge the Rel2 branch and review the
# new log.
git checkout master
# You'll be prompted for a commit message here.
git merge Rel2
git log --decorate --graph --oneline --all
### Slide: Merging approaches: Rebase
# Remove and refresh the simpleRepo
${BASE_DIR}/simple-setup.sh
cd ${BASE_DIR}/simpleRepo
# Check out master, review its log, merge Rel1 and review the log
git checkout master
git log --decorate --graph --oneline --all
git merge Rel1
git log --decorate --graph --oneline --all
# Check out Rel2
git checkout Rel2
# Rebase Rel2 onto the now-new master
git rebase master
# Review the log
git log --decorate --graph --oneline --all
### Slide: refs
# Remove and refresh the simpleRepo
${BASE_DIR}/simple-setup.sh
cd ${BASE_DIR}/simpleRepo
# Look at the contents of .git/refs/heads and one of the heads itself
ls -l .git/refs/heads/
cat .git/refs/heads/Rel2
# Tag a branch, look at the branch ref and the tag ref
git tag Rel1.0 Rel1
cat .git/refs/heads/Rel1
cat .git/refs/tags/Rel1.0
# Look at the HEAD ref
cat .git/HEAD
# Go to another clone and look at the refs for the origin remote
cd ${BASE_DIR}/devTeamDemo/javaBootcampNoEclipse.dev1
ls -l .git/refs/remotes/origin/
# .git/packed-refs contains the refs that haven't been interacted with
# yet.
cat .git/packed-refs
### Slide: Annotated tags
cd ${BASE_DIR}/simpleRepo
# Look at a "lightweight" tag
cat .git/refs/tags/Rel1.0
# What type of object is it pointing to?
git cat-file -t $(cat .git/refs/tags/Rel1.0)
# What's in the object it's pointing to
git cat-file -p $(cat .git/refs/tags/Rel1.0)
# Create an annotated tag and look at *it*
git tag -a -m "Formal release of 1.0" Rel1.0.prod Rel1
cat .git/refs/tags/Rel1.0.prod
# What type of object is it pointing to?
git cat-file -t $(cat .git/refs/tags/Rel1.0.prod)
# What's in the object it's pointing to
git cat-file -p $(cat .git/refs/tags/Rel1.0.prod)
### Slide: blame
# Look at the lines of the file on master
git checkout master
git blame information.md
# .. and on Rel1
git checkout Rel1
git blame information.md
# And on master after Rel2 has been merged in.
git merge Rel2
git blame information.md
# Slide
### Slide: Tag and commit signing
# Look at your secret keys
gpg --list-secret-keys
# Check out master and make a change to information.md
git checkout master
cat <<EOF >> information.md
An additional line for demonstrating commit-signing.
EOF
# Add and commit the change, signing the commit.
git add information.md
git commit -S<secretKeyID> -m "Update to information.md"
# Look at the commit object
git cat-file -p master
# Create an annotated tag from the Rel2 branch, signing it.
git tag -s -u <secretKeyID> -m "Release 2." Rel2.0 Rel2
# Look at the tag object
git cat-file -p Rel2.0
# Verify the signed tag and the signed commit.
git tag -v Rel2.0
git log --show-signature -1
### Slide: Git Objects &#x2013; blobs
# Generate the SHA1 of the contents of the ~/.bash_history file as
# though git would
cat ~/.bash_history | git hash-object --stdin
### Slide: Git Objects &#x2013; trees
cd ${BASE_DIR}/devTeamDemo/javaBootcampNoEclipse.dev1
# Get the tree object for the latest version of the top-level of the
# project
git cat-file -p master | grep '^tree'
# Look at the contents of that tree object:
git cat-file -p $(git cat-file -p master | grep '^tree' | sed 's/^tree //')
### Slide: Git Objects &#x2013; commits
cd ${BASE_DIR}/simpleRepo
git log --graph --all --decorate --oneline
git cat-file -p HEAD
git cat-file -p <anyOtherCommitId>
### Slide: Git Objects &#x2013; tags
# Find all the tag objects
git cat-file --batch-check='%(objecttype) %(objectname) %(objectsize)' --batch-all-objects | grep tag
git cat-file -p <anyOfTheTagObjects>
### Slide: "Content Addressable Filesystem"
# Find all the objects, and select one that refers to a file
git rev-list --all --objects
# Look at the contents of the selected object
git cat-file -p <selectedBlobObject>
# Use git to get the SHA1 of the contents of that object
git cat-file -p <selectedBlobObject> | git hash-object --stdin
# The name of the file for that object is based on the SHA1.
ls -l .git/objects/...
### Slide: The reflog
# Look at the reflog, then clear it completely and look at it again.
git reflog
git reflog expire --expire=now --expire-unreachable=now --verbose --all
git reflog
### Slide: fsck and gc
git fsck
git gc
### Slide: Useful commands
# Go into the repo where the large file had been created
cd ${BASE_DIR}/largeFileRepo
# List all the objects in increasing order of object size.
git cat-file --batch-check='%(objecttype) %(objectname) %(objectsize)' --batch-all-objects | sed -n 's/^blob //p' | sort -n --key=2
# Look for the file name associated with an object
git rev-list --objects --all | grep <blobID>
# Look for the commits that made changes to a specific file
git log --follow -- "largeInformation.md"
### Slide: Permanently removing a file from your git db
# Preserve information on the tags, as you may need this later.
for tag in $(git tag)
do
echo "${tag},$(git log --format="%H,\"%cn\",\"%ci\",\"%s\"" ${tag} | head -1)"
done | tee /tmp/tag_list.csv
# Create a local branch for all the remote branches. This does nothing
# in this demo as there is no remote.
for branch in $(git branch -r | grep -v HEAD | sed 's/\ \ origin\///')
do
git branch ${branch} origin/${branch}
done
# Check out each branch and determine the amount of space it uses
for branch in $(git branch | sed 's/^..//')
do
git checkout -q ${branch}
du -sk . | sed "s/\./${branch}/"
done | tee /tmp/branch_sizes.out
# Check out the Rel1 branch
git checkout Rel1
# Use git-filter-branch to remove the file from all the commits on the
# branch
git filter-branch --tree-filter 'rm -f largeInformation.md' --prune-empty HEAD
# Update refs (will fail for some branches)
git for-each-ref --format="%(refname)" refs/original/ | xargs -n 1 git update-ref -d
# Do the same for all the other branches
for branch in Rel2 Rel3 Rel4 master
do
git checkout ${branch}
git filter-branch --tree-filter 'rm -f largeInformation.md' --prune-empty HEAD
git for-each-ref --format="%(refname)" refs/original/ | xargs -n 1 git update-ref -d
done | tee /tmp/cleanup.out
# Clear out the reflog
git reflog expire --expire-unreachable=now --all
# Run the garbage collector on the repository
git gc --prune=now
# Run an FSCK on the repo
git fsck --unreachable --no-reflogs
# Check out each branch again and determine the amount of space it
# uses
for branch in $(git branch | sed 's/^..//')
do
git checkout -q ${branch}
du -sk . | sed "s/\./${branch}/"
done | tee /tmp/branch_sizes_post_process.out