GitHub Copilot

GitHub Copilot is supposed to suggest and complete code in­de­pen­dent­ly as an assistant. Currently, the GitHub AI is still in the testing phase and is prone to errors. As time goes on, however, it should work much more ef­fec­tive­ly.

To un­der­stand what exactly the GitHub Copilot is and what it is supposed to do, it is important to first look at GitHub. GitHub is a col­lab­o­ra­tive version control system whose US publisher has been part of Microsoft since 2018. GitHub is designed to allow large teams to work on code together and in­de­pen­dent­ly. All versions are stored and changes can be merged as desired.

Since 2021, the company now offers GitHub Copilot for some users. The AI tool is a smart assistant designed to help de­vel­op­ers create code. Initially, this works via auto-com­ple­tion. When the user starts typing in code, GitHub Copilot makes several sug­ges­tions to the user on how that code might look complete. Copilot obtains its in­for­ma­tion from publicly available code, such as different repos­i­to­ries. GitHub AI goes much further, rec­og­niz­ing numerous Internet pro­gram­ming languages such as Go, Java, JavaScript, Python, Ruby, and Type­Script. The ar­ti­fi­cial in­tel­li­gence gets smarter and smarter as time goes on and then provides better sug­ges­tions.

Con­verse­ly, this also means that GitHub Copilot is currently still very ex­pand­able. The company itself also points out that the suggested code is not yet perfect. In addition, the hit rate of the sug­ges­tions is very low so far. Users must therefore assume that the code is not yet ex­e­cutable and that some of the sug­ges­tions are even unusable. However, the copilot in Git already offers the first useful hints or truly usable sug­ges­tions.

The basis for GitHub Copilot is provided by the language pro­duc­tion system GPT-3. This was published in 2020 by OpenAI and uses deep-learning strate­gies to complete human texts or to compose its own texts. The AI uses various al­go­rithms for this, collects huge amounts of data and creates new content from it, which should hardly differ from the texts of human authors. The same applies here: The more the AI is “fed”, the better its results will be. Attempts were already made with GPT-3 to create code on the basis of learned struc­tures. Microsoft then invested massively in OpenAI and GPT-3, so that the knowledge gained can be used for GitHub Copilot.

Currently, GitHub Copilot is not yet freely available. Only a group of selected testers have the op­por­tu­ni­ty to test the AI at the moment, make sug­ges­tions, and improve the tool that way. The goal is to sooner or later turn GitHub Copilot into a com­mer­cial program used by de­vel­op­ers for their daily work on new software. However, it is not yet known when the AI will be ready. During the learning and testing phase, those in­ter­est­ed can only get a first glimpse. Visual Studio Code, Neovim, and JetBrains IDEs such as PyCharm and IntelliJ IDEA are currently supported.

While the initial reports are promising, GitHub Copilot still seems to be far from market-ready. The overall hit rate is not yet par­tic­u­lar­ly high and the quality of the sug­ges­tions is also clearly ex­pand­able. For the most part, the code is not yet usable and leads to errors in many cases. The quality of a future com­mer­cial release will depend heavily on how well the AI learns and the quality of the source code provided to it. Errors in the source material are currently still taken over by GitHub Copilot just as un­sus­pect­ing­ly as unclean syntax. After the learning phase, the results should also get better.

In addition to the afore­men­tioned problems with in­ap­pro­pri­ate sug­ges­tions or ex­pand­able syntax, there are also dis­cus­sions about the basic error-proneness of the codes that currently arise with GitHub Copilot or could arise in the future. Since the basis through which the AI is supposed to learn is often faulty or at least untested, the end result is also too often uncertain. Although it is pointed out that all input provided by the AI is to be verified, it is at least ques­tion­able whether this can actually help daily work in the long run. In previous tests, the code from GitHub Copilot often performed poorly.

Some de­vel­op­ers also fear that using Copilot in Git could po­ten­tial­ly lead to copyright in­fringe­ment should the AI simply take over entire blocks of code. While there are different fair use rules; whether an AI’s learning successes fall under them is at least debatable. This is all the more true if GitHub Copilot could also be used for com­mer­cial purposes in the future. The company itself explains that right now only a few source codes are taken over com­plete­ly or partially unchanged. With greater learning successes, this figure is expected to drop even further.

Currently, GitHub Copilot is still a gimmick whose added value is very man­age­able. However, once the AI has learned more, it could take a lot of work off de­vel­op­ers’ shoulders. On the one hand, it could show al­ter­na­tive solutions and provide suitable syntax examples without a tedious search in different doc­u­men­ta­tion. On the other hand, it should add in­di­vid­ual code blocks in­de­pen­dent­ly at some point and therefore con­tribute time-consuming lines. Although this would make the work easier, a certain basic knowledge will still be necessary for de­vel­op­ment. It will probably be a long time before an AI writes code in­de­pen­dent­ly.

GitHub Copilot is an obvious idea that could someday be a natural part of working with source code. The idea of an attentive assistant that takes over smaller tasks and points out possible errors is quite promising. Currently, however, the AI is still very far away from this role. The current test phase is only a first step in this direction and the error rate is therefore high, as expected. It is not yet possible to reliably say when GitHub Copilot will actually be available to all in­ter­est­ed parties. However, a first step has been made with the test phase.