GIT

Git – Useful Commands

I am posting  the git commands, which are very useful for git users.

##Initial configuration Configure user details only for the first time

git config --global user.name "Swami Reddy MR"
git config --global user.email "swamireddy@gmail.com"

To check your git configuration

git config --list

##Branches List all branches in the local repo

git branch

List all branches including remote-tracking branches

git branch -r

List all branches (including the ones at remote)

git branch -a

Creating your own branch my_branch, with content same as your current branch

git branch my_branch

Switch to the newly created branch

git checkout my_branch

Execute the above two commands in one line:

git checkout -b my_branch

to pull a branch only_remote which exists at github (at remote swamireddy but not in local repo. (More about git remote update later)

git remote update
git checkout -b only_remote swamireddy/only_remote

Remotes

Create a new remote swamireddy which will track cinder repository by user swamireddy

git remote add swamireddy https://github.com/swamireddy/cinder.git

If this is your personal repo, you can append the username in the remote as shown. After that, every time you push to the origin, github will not ask for your username but only password.

git remote add swamireddy https://swamireddy@github.com/swamireddy/cinder.git

Delete the remote swamireddy

git remote rm swamireddy

List all the remotes

git remote

The above command only shows the names of remotes. To also check the links to the remotes:

git remote -v

Now this is very important command.

git remote update

This will contact the git server, and will update the local repository with ALL the content at ALL those remotes. The interesting part is, nothing will actually change. That is, no more new branches (which got created at the server after you last pulled from the server) were created, and the existing branches are also not updated. BUT, all the content goes into the magic .git directory. After you’ve run this command, when you create a branch only existing at remote, the local git repo will not contact the server but will fetch all the contents from within the repository.

To take a new branch from remote, and create a new branch having contents of that remote branch. (You might need to run ‘git remote update’)

git checkout -b stable/cinder origin/stable/cinder

Push

Pushing this newly created branch to remote swamireddy

git push swamireddy my_branch

If you want to push the local branch my_branch with a different name to remote, say my_remote_branch:

git push swamireddy my_branch:my_remote_branch

To create an association with the remote (only first time):

git push -u origin my_branch

or

git push --set-upstream origin my_branch

Pull

Git pull is nothing more than a macro that does git fetch and git merge, in that order. The common syntax to pull from branch remote_branch located at remote remote_name to the current active branch in the local repo:

git pull remote_name remote_branch

Oops! I didn’t intend to do that!

####To undo last commit. This will just undo the commit, but will keep changes, so that you can modify the files and commit again

git reset --soft HEAD^

####To not keep the uncommitted changes. This command will wipe off all the changes which are not committed. Very useful when you made some changes but dont want to commit it. All the more useful when you pulled something but everything became a mess (possibly due to a merge conflict, or pulling to/from a different branch!)

git reset --hard

####To delete a branch testing from github server

git push swamireddy --delete testing

(NOT git push swamireddy --delete swamireddy/testing)

####Change author of the last commit

git commit --amend --author="Swami Reddy MR <swamireddy@gmail.com>"

You committed some changes, but then you realised you wanted to add this one line to the commit. In such a case just add that one line, and run this command to have this last change incorporated into that previous commit. This command also gives you an option to change the commit message.

git commit -a --amend

Show me the money

To see the patch of the last commit without undoing the last commit

git show

To see the changes introduced by any earlier commit with commit id c5bb6d821e10ca8f114fa0b3b0149bc8b7257a92

git show c5bb6d821e10ca8f114fa0b3b0149bc8b7257a92

To see the latest changes you made – the changes which have not been staged to be committed

git diff

You can redirect the output from the above three commands to a file, to create corresponding patch file.

Patching in git

Check the status of patch. How many lines changed, etc

git apply --stat patchfile

Check if the patch can be applied. If no output or no error, patch can be applied safely

git apply --check patchfile

Apply patch with signing-off (better way)

git am --signoff < patchfile

Normal way of applying patch

git apply patchfile

##Miscellaneous

To pull only specific files from another branch: (here, assuming that we have two branches, ‘test’, and ‘master’, and currently we are on ‘master’ branch. If you want to pull ##only ‘testfile.py’ file from ‘test’ branch to ‘master’ branch, do this:)

git checkout test testfile.py1

If your master changed while you were working on your topic_branch, and if you want to merge the master’s changes, and also get an option to squash the changes you made on topic_branch:

git checkout master
git pull origin master
git checkout topic_branch
git rebase -i master

Undo last commit at the remote repo

(you can use git revert, but don’t know how exactly it works)

Clone a specific branch git repository from github

git clone -b stable/essex https://github.com/openstack-dev/devstack.git

Squash several commits into one single commit

Checkout a previous commit with SHA commit id cff2580ad7bc16934b72dd9be7463eb090b31d55 from the current branch to a new branch ‘new_branch’

git checkout -b new_branch cff2580ad7bc16934b72dd9be7463eb090b31d55
Advertisements
Ceph

Ceph-mons: List all configuration settings

Really helpful to use a single command to list all the configuration flags on ceph monitor node.

Use the below command to list all the ceph monitor’s configuration
#  ceph daemon /var/run/ceph/ceph-mon.*.asok config show | more

The output would be long,  so use ‘more’ or grep for a specific value in case you know what you are looking for.

NOTE:  Admin socket for the monitor will be in the path /var/run/ceph/ceph-mon.<hostname-s>.asok  by default.

Use the below command to get specific  (here flag: rgw_enable_usage_log ) the ceph monitor’s configuration
#  ceph daemon /var/run/ceph/ceph-mon.*.asok config get rgw_enable_usage_log
Use the below command to set specific  ceph monitor’s configuration
#  ceph daemon /var/run/ceph/ceph-mon.*.asok config set rgw_enalbe_usage_log true
Uncategorized

Ceph: How to add the SSD journal with dm-crypt enabled

Here am sharing the steps for adding a journal with dm-crypt enabled.

Steps:

Create a key using the journal partition’s uuid das below;

dd  bs=key-size count=1 if=/dev/urandom of=/etc/ceph/dmcrypt-keys/<journal_partition uuid>

For ex:

  dd bs=512 count=1 if=/dev/urnadom of=/etc/ceph/dmcrypt-keys/uuid
How to find the jounral partition uuid:
 #ls -l /dev/disk/by-partuuid/

Now, create use the below command to do dm-crypted partition:

  1. cryptsetup -v –cipher aes-xts-plain64 –key-size 512 –key-file  /etc/ceph/dmcrypt-keys/uuid luksFomat  <journal partition>
cryptsetup -v --cipher aes-xts-plain64 --key-size 512 --key-file  /etc/ceph/dmcrypt-keys/uuid luksFomat  /dev/sdc5

2.  cryptsetup -v open –type luks <jounral partition> <uuid>

#cryptsetup -v open --type liks /dev/sdc5   uuid

Now, stop the OSD and flush the current journal.

#systemctl stop ceph-osd@<id>
#ceph-osd --flush-journal -i <id>

Now, go to /var/lib/ceh/osd/ceph-<id>/   and move the current journal and journal_dmcrypt to old as below:

# mv jounral jounral.old
# mv journal_dmcrypt jounral_dmcrypt.old

Now create soft links for the newly created journal and jounral_dmcrypt files as below:

# ln -s /dev/disk/by-partuuid/<uuid> ./journal_dmcrypt
# ln -s /dev/mapper/<uuid>  ./jounral

NOTE: the above files, permission may be required to change with ceph:ceph as below:

 # chown ceph:ceph journal jounral_dmcrypt

Now, create osd journal as below:

# ceph-osd --mkjournal -i <id>

Now, start the osd as below:

#systemclt start ceph-osd@<id>

OSD should be up and in with new journal.

 

 

Ceph

Ceph: mon is down and/or can’t rejoin the quorum

Sometimes, we have seen that a Ceph mon down and could not rejoin the ceph mon quorum, even though that specific ceph mon is up and running (along with ceph-mon process is also up and running).

A quick solution as below:

As one ceph mon is down and out of quorum, then its safe to remove the down mon node from the quorum with below steps:

Pre-requisites: Connect to ceph mon node (or controller node, where down ceph mon is installed and check if its running or not using “ps -ef | grep ceph-mon”. [ If its running with nonresponsive, then stop/kill this process]. The output of this “ps” command should be empty.

Remove the ceph mon data directory:
  # rm -f /var/lib/ceph/mon/ceph-node-x

Create new auth key
    # ceph auth get mon. -o key.txt

Get a copy of mon map (like monmap.bin)
   # ceph mon getmap -o monmap.bin

Inject the down and out of quorum ceph mon into ceph monmap

   # ceph-mon -i node-x --mkfs --inject-monmap map.bin --keyring key.txt

Now, start the ceph mon service
    # start ceph-mon id=node-x
        or
    # systemctl start ceph-mon@id

Remove the monmap.bin and key.txt files.

 

Ref: 
http://docs.ceph.com/docs/master/rados/troubleshooting/troubleshooting-mon/#recovering-a-monitor-s-broken-monmap
Ceph

Ceph: RBD fails to map an image to a block device

When I tried to map an image to block device, I got the below error using ceph “rbd map” command:

#rbd create image01 --size 1024 -p rbdbench
#rbd map image01 -p rbdbench --name client.admin
rbd: sysfs write failed
RBD image feature set mismatch. You can disable features unsupported by the
kernel with "rbd feature disable".
In some cases useful info is found in syslog - try "dmesg | tail" or so.
rbd: map failed: (6) No such device or address

This can be fixed by passing appropriate features enabled either manually or set he default feature set in the Ceph  (/etc/ceph/ceph.conf) configuration.

 To fix the above issue: To resolve this issue, edit

Option#1:  Added the line in ceph config i.e /etc/ceph/ceph.conf file:

rbd default features = 3

Option#2: Alternatively, we can enable the “layering” feature by adding “--image-feature layering" flag, when we create the image. For example:

#rbd create image02 --size 1024 --pool rbdbench --image-feature layering
# rbd map image02 -p rbdbench --name client.admin
/dev/rbd1

 

Uncategorized

Ceph: Increase OSD start timeout

For encrypted OSDs, need to increase OSD start timeout from default value “300” to “900”, to start the OSD. How can we do this? just follow the below 2 steps, which can be achieved:

#cp /lib/systemd/system/ceph-disk@.service /etc/systemd/system/
#sed -i "s/CEPH_DISK_TIMEOUT=300/CEPH_DISK_TIMEOUT=900/" /etc/systemd/system/ceph-disk@.service

Now restart the OSDs:

# systemctl restart ceph-osd\*.service ceph-osd.target
Ceph

Ceph: Whats new in this release

Here is the quick list of new features/functionalities added in ceph releases.

Please find the more details on each release with LTS and its retirement dates  Ceph release

Luminous (RC) (2017) – Release notes

BlueStore backend for ceph-osd is now stable.
There is a new daemon, ceph-mgr, is a required part of any Ceph deployment. 
Multiple active MDS daemons is now considered stable.
The ceph status (ceph -s) command has a fresh look.
S3 bucket lifecycle API has been added.

Kraken (2016) – Release notes

AsyncMessenger supported
BlueStore backend declared as stable
RGW : metadata indexing via Elastic search, index resharding, compression
S3 bucket lifecycle API, RGW Export NFS version 3 throw Ganesha
Rados support overwrites on erasure-coded pools / RBD on erasure coded pool (experimental)

Jewel (2016)  – Release notes

CephFS declared as stable
RGW multi-site re-architected (Allow active/active configuration)
AWS4 compatibility
RBD mirroring supported
BlueStore (experimental)
Support for NFS version 3 has been added to the RGW NFS gateway.

Infernalis (2015) –  Release notes

Erasure coding declared as stable and support many new features
New features for Swift API (Object expiration,…)
Systemd

Hammer (2015) – Release notes

RGW object versioning, bucket shardingversioning
Crush straw2

Giant (2014) – Release notes

LRC erasure code
CephFS journal recovery, diagnostic toolsCephFS journal recovery, diagnostic tools

Firefly (2014) – Release notes

Erasure coding
Cache tieringtiering
Key/value OSD backendbackend
Standalone radosgw (with civetweb)

Emperor (2013) – Release notes

Multi-datacenter replication for the  radosgw
Improved usability
Improved crc32c performance
Validate S3 tokens against Keystone

Dumpling (2013) – Release notes

Multi-site support for radosgw
RESTful API endpoint for Ceph cluster administration
Object namespaces in librados.

 

Ref: 
http://docs.ceph.com/docs/master/release-notes/
http://cephnotes.ksperis.com/blog/2017/01/06/main-new-features-in-the-latest-versions-of-ceph/