How to create a commit and push into repo with GitHub API v3?

Posted on

Question :

How to create a commit and push into repo with GitHub API v3?

I want to create a repository and Commit a few files to it via any Python package. How do I do?

I do not understand how to add files for commit.

Answer #1:

You can see if the new update GitHub CRUD API (May 2013) can help

The repository contents API has allowed reading files for a while. Now you can easily commit changes to single files, just like you can in the web UI.

Starting today, these methods are available to you:

Answered By: VonC

Answer #2:

Here is a complete snippet:

def push_to_github(filename, repo, branch, token):
    url="https://api.github.com/repos/"+repo+"/contents/"+filename

    base64content=base64.b64encode(open(filename,"rb").read())

    data = requests.get(url+'?ref='+branch, headers = {"Authorization": "token "+token}).json()
    sha = data['sha']

    if base64content.decode('utf-8')+"n" != data['content']:
        message = json.dumps({"message":"update",
                            "branch": branch,
                            "content": base64content.decode("utf-8") ,
                            "sha": sha
                            })

        resp=requests.put(url, data = message, headers = {"Content-Type": "application/json", "Authorization": "token "+token})

        print(resp)
    else:
        print("nothing to update")

token = "lskdlfszezeirzoherkzjehrkzjrzerzer"
filename="foo.txt"
repo = "you/test"
branch="master"

push_to_github(filename, repo, branch, token)
Answered By: Martin Monperrus

Answer #3:

Github provides a Git database API that gives you access to read and write raw objects and to list and update your references (branch heads and tags). For a better understanding of the topic, I would highly recommend you reading Git Internals chapter of Pro Git book.

As per the documentation, it is a 7 steps process to commit a change to a file in your repository:

  1. get the current commit object
  2. retrieve the tree it points to
  3. retrieve the content of the blob object that tree has for that particular file path
  4. change the content somehow and post a new blob object with that new content, getting a blob SHA back
  5. post a new tree object with that file path pointer replaced with your new blob SHA getting a tree SHA back
  6. create a new commit object with the current commit SHA as the parent and the new tree SHA, getting a commit SHA back
  7. update the reference of your branch to point to the new commit SHA

This blog does a great job at explaining this process using perl. For a python implementation, you can use PyGithub library.

Answered By: Jai Pandya

Answer #4:

Solution using the requests library:

NOTES: I use the requests library to do the calls to GitHub REST API v3.

1. Get the last commit SHA of a specific branch

# GET /repos/:owner/:repo/branches/:branch_name
last_commit_sha = response.json()['commit']['sha']

2. Create the blobs with the files content (encoding base64 or utf-8)

# POST /repos/:owner/:repo/git/blobs
# {
#  "content": "aGVsbG8gd29ybGQK",
#  "encoding": "base64"
#}
base64_blob_sha = response.json()['sha']

# POST /repos/:owner/:repo/git/blobs
# {
#  "content": "hello world",
#  "encoding": "utf-8"
#}
utf8_blob_sha = response.json()['sha']

3. Create a tree which defines the folder structure

# POST repos/:owner/:repo/git/trees/
# {
#   "base_tree": last_commit_sha,
#   "tree": [
#     {
#       "path": "myfolder/base64file.txt",
#       "mode": "100644",
#       "type": "blob",
#       "sha": base64_blob_sha
#     },
#     {
#       "path": "file-utf8.txt",
#       "mode": "100644",
#       "type": "blob",
#       "sha": utf8_blob_sha
#     }
#   ]
# }
tree_sha = response.json()['sha']

4. Create the commit

# POST /repos/:owner/:repo/git/commits
# {
#   "message": "Add new files at once programatically",
#   "author": {
#     "name": "Jan-Michael Vincent",
#     "email": "JanQuadrantVincent16@rickandmorty.com"
#   },
#   "parents": [
#     last_commit_sha
#   ],
#   "tree": tree_sha
# }
new_commit_sha = response.json()['sha']

5. Update the reference of your branch to point to the new commit SHA (on master branch example)

# POST /repos/:owner/:repo/git/refs/heads/master
# {
#     "ref": "refs/heads/master",
#     "sha": new_commit_sha
# }

Finally, for a more advanced setup read the docs.

Answered By: Arthur Miranda

Answer #5:

I’m on Google App Engine (GAE) so beside of python, I can create a new file, update it, even delete it via a commit and push into my repo in GitHub with GitHub API v3 in php, java and go.

Checking and reviewing some of the available third party libraries to create like the example script that presented in perl, I would recommend to use the following:

As you aware, you can get one site per GitHub account and organization, and unlimited project sites where the websites are hosted directly from your repo and powered by Jekyll as default.

Combining Jekyll, Webhooks, and GitHub API Script on GAE, along with an appropriate GAE Setting, it will give you a wide possibility like calling external script and create a dynamic page on GitHub.

Other than GAE, there is also an option run it on Heroku. Use JekyllBot that lives on a (free) Heroku instance to silently generates JSON files for each post and pushing the changes back to GitHub.

Answered By: Chetabahana

Answer #6:

Based on previous answer, here is a complete example. Note that you need to use POST if you upload the commit to a new branch, or PATCH to upload to an existing one.


    import whatsneeded
    
    GITHUB_TOKEN = "WHATEVERWILLBEWILLBE"
    
    def github_request(method, url, headers=None, data=None, params=None):
        """Execute a request to the GitHUB API, handling redirect"""
        if not headers:
            headers = {}
        headers.update({
            "User-Agent": "Agent 007",
            "Authorization": "Bearer " + GITHUB_TOKEN,
        })
    
        url_parsed = urllib.parse.urlparse(url)
        url_path = url_parsed.path
        if params:
            url_path += "?" + urllib.parse.urlencode(params)
    
        data = data and json.dumps(data)
        conn = http.client.HTTPSConnection(url_parsed.hostname)
        conn.request(method, url_path, body=data, headers=headers)
        response = conn.getresponse()
        if response.status == 302:
            return github_request(method, response.headers["Location"])
    
        if response.status >= 400:
            headers.pop('Authorization', None)
            raise Exception(
                f"Error: {response.status} - {json.loads(response.read())} - {method} - {url} - {data} - {headers}"
            )
    
        return (response, json.loads(response.read().decode()))
      
    def upload_to_github(repository, src, dst, author_name, author_email, git_message, branch="heads/master"):
        # Get last commit SHA of a branch
        resp, jeez = github_request("GET", f"/repos/{repository}/git/ref/{branch}")
        last_commit_sha = jeez["object"]["sha"]
        print("Last commit SHA: " + last_commit_sha)
    
        base64content = base64.b64encode(open(src, "rb").read())
        resp, jeez = github_request(
            "POST",
            f"/repos/{repository}/git/blobs",
            data={
                "content": base64content.decode(),
                "encoding": "base64"
            },
        )
        blob_content_sha = jeez["sha"]
    
        resp, jeez = github_request(
            "POST",
            f"/repos/{repository}/git/trees",
            data={
                "base_tree":
                last_commit_sha,
                "tree": [{
                    "path": dst,
                    "mode": "100644",
                    "type": "blob",
                    "sha": blob_content_sha,
                }],
            },
        )
        tree_sha = jeez["sha"]
    
        resp, jeez = github_request(
            "POST",
            f"/repos/{repository}/git/commits",
            data={
                "message": git_message,
                "author": {
                    "name": author_name,
                    "email": author_email,
                },
                "parents": [last_commit_sha],
                "tree": tree_sha,
            },
        )
        new_commit_sha = jeez["sha"]
    
        resp, jeez = github_request(
            "PATCH",
            f"/repos/{repository}/git/refs/{branch}",
            data={"sha": new_commit_sha},
        )
        return (resp, jeez)

Answered By: Chmouel Boudjnah

Leave a Reply

Your email address will not be published.