In our hectic world you may write programs in a huge variety of languages: C#, C/C++, F#, Haskell, Visual Basic and the list seems endless. The common factor among them is that every program, regardless of language, will be compiled before it becomes proper working machine code. Exploiting this common trait may revolutionize our way of thinking about programming, and the first technology to try this is here, under the code name of Phoenix.
Contributed by Gabor Bernat Rating: / 1 March 04, 2009
Phoenix is an innovative framework that tries to create a central compiler system that can handle all of the imaginable language compilers, and even offers the option of writing your own compiler for your own language.
The Phoenix Logo - Image Courtesy Of Microsoft
Of course, it also completes a couple of other tasks, like letting you extend the compiler for the languages already existing, and as the upper image states: generate, optimize and analyze your code.
However, before we go any further let me sum up concisely the topic of this article. Just as the title suggests, it is all about presenting the Phoenix Compiler Framework. I will divide the whole process into two articles. During the first one (what you are already reading), I will present the framework's main concept and what you can do with it.
Within an article that will appear later titled "Low Level Code Optimization in Phoenix,"I will go into the more complicated details of how Phoenix accomplishes its work. This will let you observe how the compiler performs and to comprehend the difficulties faced by producing machine code from what you type inside the computer.
The development team made a couple of interesting videos related to this that appeared on channel 9, however, for those of you that do not want to watch an hour-long video for a brief overview, let's get moving.
Kang Su Gatlen announced the project back in 2004, declaring that it would be a next generation compiler that will represent the basis of code generation in the future for all of the Microsoft's applications.
The Phoenix is a joint project of the Microsoft Research, Common Language Runtime (CLR) and Visual C++ teams. So the Research segment works with the development team to create a compiler that can be used by university professors to teach compiler principles, and also by developers for their code.
The Phoenix platform overview - Image Courtesy Of Microsoft
This is an Open Framework, so anyone interested in it can use it. Microsoft will certainly use it; in fact, the company intends to compile all of their products (OS, Office, Visual Studio, and so forth) with it.
Nevertheless there are a couple of situations and structures that, while discussed in theory, may come up that are absurd in practice -- when you want, for instance, to compile a million lines of long code. Therefore, to develop something that can handle this, the entire framework is pluggable and modular.
In practice, this means that you can replace every part of the framework with your own specialized code, or you can just extend it where you want to. With time, an SDK (Software Development Kit) emerged, providing program analysis and transformation regardless of the language used.
At the heart of Phoenix is the IR. This has multiple levels, however it's really nothing more than a detailed explanation of what your program is trying to accomplish. Here every line of your code snippet is expanded and reduced to a level that breaks the whole application into a linear data flow. IR components such as flow graphs, region graphs, and multi-model exception handling ensure that representations are robust and correct.
The Phoenix Compilers Architecture - Image Courtesy Of Microsoft
At this point, you will see only a collection of labels and "gotos." A good point for Phoenix is that it will generate the same code for all languages. Therefore, merging a managed language and a native language like F# and C++ is no problem for the framework. It does not have to know what it analyzes, it just takes the IR flow and makes from it machine code.
For instance, imagine you have a function inside MSIL, C++ or any other language. You cannot tell what effect it has without building the program stack and seeing where exactly it stays, what effect it has on its environment and how it connects with another function. All of this, of course, should be done inside the code that the machine uses to execute the application.
In Phoenix IR in between two functions there is a relation (such as one gives a value as a parameter to a second) which will be explicit and direct. You can see how the value will be generated in one place and get consumed later on. Inside the IR exists a lot of information just like this, so any tool can fetch it.
What is even more brilliant is that Phoenix can also raise the code from the binaries. This means that it is possible to raise an exe from the binary level to IR, extend it with some additional functionality and afterward just recompile it and produce the new machine code.
Still, it is crucial to observe that it won't decompile the program. That is more difficult. You may produce code that does the same thing as your original and is valid -- however, you cannot possibly know if the loop was a "for" or a "while," and the upshot of the two different loops won't be the same. Phoenix is in essence a code generator for various platforms like the X86, X64, AMR, and so forth.
To understand Phoenix more deeply, let us take, for instance, the exception handling issue. A compiler should probably represent the ultimate in paranoia. It should take into account all of the things that might go wrong inside a program, even though many of them will never happen.
However, the compiler cannot possibly know this, if the language lets it happen; and it is possible the compiler should produce code for it to handle the situation in case it does happen. On the other hand, this code may drastically slow down your program, or you may not want to waste time with treating problems that will never happen.
For example, a pointer may point to all of the things you can imagine in theory; however, when you code, most of the time it will point only to a couple of objects. In Phoenix, you can create a plug-in for a situation that will tell the compiler to be more optimistic and ignore exceptions like this.
Of course this will be a source of bugs if it does happen, but you've gained some development speed and also an optimal application that might run faster. The plug-in based model will allow you to do much more interesting things, such as creating new language rules for your company.
For example, assume that one day you decide that you no longer want recursive functions inside the application you're developing. Of course, you can ban the use of it afterward; however, one of your employees might slip one in, in a masked form.
With Phoenix, you can disallow this at the compiler level. Just write a plug-in that observes a recursive function and throws a compiler error at it that says "You should not use recursive functions." Now this is just a simple example, but it can be extended to a whole set of rules. You may read a book on how C++ code should be written, and the next day write a compiler that will compel users to write in compliance.
Some problems you may be able to resolve automatically, like variable initialization; however, for others this is not possible. One of the best examples involves SECURE_CRT.
In 2005, Microsoft released a set of secure functions and deprecated the previous ones. Now for backward compatibility, these were maintained, so everybody can use them. With Phoenix, on the other hand, you may introduce the rule that nobody is ever allowed to bring the deprecated functions into play.
Even better, you can observe when someone does something similar, such as a strcat (for instance) and throw an error that says this user is trying to use a construction that is like a strcat. It is possible to detect all kinds of things.
Now you can even provide your errant programmer with a link to how he should resolve the problem and why that construction is wrong/unsafe to use. This will not provide an automatic solution; however, if you point the developer in the right direction, that is already a huge help. Moreover, if the developers are writing their compiler, they can aim it more in line with their demands.
It is possible also to allow something stupid on purpose because it won't be used anywhere else except for that specific function, for instance, or you can "trust but verify" (meaning that as long as you do not use it elsewhere, the compiler will not comment).
Another interesting trait of Phoenix is its modularity. This means that you can replace any part of it with your own and use the others. When replacing the File Writer part, you can build your application for a new platform. Substituting the File Reader part will let you turn your coding language into Phoenix IR (Intermediate Representation), and afterward you can build the machine code for all platforms.
The "Tiger language" creation- Image Courtesy Of Microsoft
So why is Phoenix better than the rest of the compilers that are already available? For starters, it is internally multi-threaded. Therefore, it will compile your application using multiple cores if they exist. During preliminary testing on an eight-core processor, your application will compile around five times faster than it would on a single core CPU (this is a significant difference if you compile for three to four hours).
Secondly, you can extend the compiler with analysis and transformation, as previously described. In the future when a new hardware product is launched, implementing a new technology, you no longer have to wait for the compiler developer team to implement support for it.
Now a hardware manufacturer can launch a Phoenix plug-in and let you take full advantage of it the next day. Impressive, is not it?
The Phoenix Framework usability - Image Courtesy Of Microsoft
Phoenix can also optimize your code and analyze it in more depth where it is needed with the help of the Profile Guided Optimization. This is in essence an instrumentation phase that will observe how your code is used, what part of it is used more heavily, and then concentrate on the key parts to optimize and analyze it down to its bones. This will make sure that the key parts of the program take priority, and you may build separate parts for separate usages.
As for more tools to help the developer, Andy Ayers, architect of Phoenix, presented an option called Dynamic Slicing. This comes in handy when you are debugging and, for example, observe that something has a value of zero. Now normally you should go back and observe systematically how it turns into zero. However, with the new option you can highlight the lines where that variable was used and changed, making it easier to look over just those specific lines. The picture below presents this:
You can observe how and why the sum at the break point has that specific value. This is a handy tool indeed. The Phoenix Compiler Framework is available for download at this address. It is still in the development phase, so do not expect too much yet; nevertheless, they are slowly making progress.
In its basics, this covers everything about Phoenix. At least, it covers the part that is most interesting to developers, the part that everyone will use. Come back next week to find out more about the back-end section, which we already have some inkling of its importance. Thank you for reading my article and I invite you to jot down your thoughts about this new framework here on the blog or over at our friendly forums at Dev Articles or Dev Hardware. Until next time, Live With Passion!
Low-Level Code Optimization in Phoenix
