Compiz port to GLES2 : Supporting the Unity Desktop

The second part of my “Compiz port to OpenGL ES” task was to make the whole thing work seemlessly with Canonical’s brand new Unity desktop. Take a look at the Unity Desktop that appeared originally in Ubuntu 11.04 Natty Narwhal :

This change, of course, promised to bring its share of problems, mainly for 2 reasons:

• Unity displays its GUI components/Widgets using the “Nux” toolkit, which is OpenGL-based. I knew Nux made extensive use of framebuffers, which compiz-gles depended on as well, so ouch.. possible interference ahead!
• The Unity/Compiz running scheme is very peculiar : Unity runs as a Compiz plugin (the “Unityshell” plugin).

Let’s take a look at the main challenges those elements posed to this task:

1) Unity depending on NUX

This, at first, wouldn’t have to be such a big deal because after all, stacking openGL calls at different software layers is a common thing. The problem here though, is that Nux made unsafe uses of framebuffer Objects (FBO). By “unsafe”, I mean that the code was not carefully unbinding FBOs back to their previous owners after use… so any caller (like compiz !) trying to nest Unity/Nux drawings in its own FBO just couldn’t do it ! This “unsafe” FBO usage comes from a “standalone” point of view and is somewhat not compatible with the compiz plugin scheme.

So what I and Jay Taoko came up with is a new nux API :

This new API lets us manage a new so-called “reference frame buffer” that allows for FBO nesting 🙂  Here it is :

=== modified file 'Nux/WindowCompositor.h'
--- Nux/WindowCompositor.h    2011-12-29 18:06:53 +0000
+++ Nux/WindowCompositor.h    2012-01-05 04:00:19 +0000
@@ -175,6 +175,23 @@
     //====================================  
   public:
+    /*!
+        Set and external fbo to draw Nux BaseWindow into. This external fbo will be
+        restored after Nux completes it rendering. The external fbo is used only in embedded mode. \n
+        If the fbo_object parameter 0, then the reference fbo is invalid and will not be used.
+
+        @param fbo_object The opengl index of the fbo.
+        @param fbo_geometry The geometry of the fbo.
+    */
+    void SetReferenceFramebuffer(unsigned int fbo_object, Geometry fbo_geometry);
+
+    /*!
+        Bind the reference opengl framebuffer object.
+
+        @return True if no error was detected.
+    */
+    bool RestoreReferenceFramebuffer();
+
     ObjectPtr<IOpenGLFrameBufferObject>& GetWindowFrameBufferObject()
     {
       return m_FrameBufferObject;
@@ -561,6 +578,10 @@
     int m_TooltipX;
     int m_TooltipY;

+    //! The fbo to restore after Nux rendering in embedded mode.
+    unsigned int reference_fbo_;
+    Geometry reference_fbo_geometry_;

All this landed in Nux 2.0 series as a big “Linaro” merge on 2012-01-06.  Take a look !  It’s still upstream in more recent versions!

2) Unity running as a Compiz plugin

It may be weird at first to think of Unity as a compiz plugin, but let’s think about it for a minute. When running compiz as our Window Manager, which makes intensive use of OpenGL ES calls, it’s not so foolish to make Unity a plugin because Unity is also making intensive usage of openGL primitives and we’ve got to find a way to coordinate the two components in a graceful manner. One manner is to run Unity as a compiz plugin (the “unityShell” plugin, that is). That way, Unity declares a series of callbacks knowed to Compiz (like glPaint, glDrawTexture, preparePaint, etc…) and compiz calls them at the right time. Every drawing operations are done, every framebuffers are filled at the right time, in the right order, in a graceful manner with the other plugins. On second thought, this decision from Canonical to write Unityshell is not so foolish after all 😛