In this post, we're going to analyze the design-space of implementing a GUI on-top of any real-time rendering loop.
Whenever I make low-level rendering system, I find myself re-implementing basic GUI elements such as textboxes, buttons, sliders, checkboxes, etc... in slightly different ways. This is pretty inefficient and, I'm sort of sick of doing these different architectural "experiments", so I want to implement a GUI in a nice canonical fashion and document it here for future-me and friends.
Starting With Event-Handling
Imagine we want to render a simple scene. Here we will just consider a background and two foreground elements. In this case it'll be a beautifully illustrated idyllic mirth with bird and insect sprites as our main test elements. We'll use some exquisite art assets I have on hand.
 | |
| beauty incarnate with hitboxes |
Now what is GUI-like about this? In the simplest off user-interaction,we would just test whether or not we clicked on the bird or cockroach sprites, our mouse event would give things like whether the left-mouse-button is down and the $(x,y)$ position of the cursor, and we would just check for whether the left-mouse-button is down and the cursor's coordinates intersect with the hitbox for either of these entities. At some point we will want to extend this approach for all kinds of other elements.
Starting with the rendering loop, most real-time rendering frameworks the main drawing loop looks like the following:
while(true)
{
poll();
update(dt);
draw();
float dt = elapsedTime();
}
This is the lowest level of abstraction we'll consider this post. Each frame we get the user's inputs and update our simulation based on it, then interpolate any motion with our dt for smoothness. This happens in about ~33 ms per loop to give us the 30fps experience needed to make our brains detect motion.
poll() checks new input states/events, update() goes through each pending motion/action of the on-screen objects, and draw() draws our graphics on the screen.
The most basic implementations of these functions would look something like:
void poll()
{
MouseEvent mouseEvt;
//if there was no mouse update, skip this
if (!pollMouse(mouseEvt))
return;
int x = mouseEvt.x, y = mouseEvt.y;
//is the left mouse being clicked this frame?
bool lclick = mouseEvt.leftMouseClick;
if (lclick && birdIntersect(x,y))
{
animateBird();
}
if (lclick && roachIntersect(x,y))
{
animateRoach();
}
if (lclick && BGIntersect(x,y))
{
animateBG();
}
}
void update(dt)
{
updateBG(dt);
updateRoach(dt);
updateBird(dt);
}
void draw()
{
//gets the buffer we draw on
Canvas& ctx = getGraphicsContext();
drawBG(ctx);
drawRoach(ctx);
drawBird(ctx);
}
Now let's remember our fundamental problem: what would be the "best" way to define clicks on the elements inside of this? Given our synchronous update() $\rightarrow$ draw() loop, we need to pass the information regarding a mouse update to the appropriate event
void poll()
{
//...
//...on top of all we had before...
//...
//adding GUI elements
if (lclick && GUIFormIntersect(x,y))
{
//start testing intersection for all other buttons...
}
//more GUI elements
if (lclick && GUIForm2Intersect(x,y))
{
//start testing intersection for all other buttons...
}
if (lclick && ButtonIntersect(x,y))
{
activateButton();
}
}
void update(dt)
{
updateBG(dt);
updateRoach(dt);
updateBird(dt);
updateFormGUI(dt);
updateForm2GUI(dt);
updateButton(dt);
}
void draw()
{
Canvas& ctx = getGraphicsContext();
drawBG(ctx);
drawRoach(ctx);
drawBird(ctx);
drawFormGUI(ctx);
drawForm2GUI(ctx);
drawButton(ctx);
//...
//potentially more
//...
}
Here are some killer™ problems with this approach:
- Most glaringly, for each new entity/GUI-element on the screen, we have to manually add its update(), handle() and draw() methods in the appropriate section according to its display and event priority. This gets old fast.
- If we have overlapping entities and we click on the overlapping region, then are "clicked" on. This may not be desirable in some cases and manually checking for that exception would suck.
- Regardless of points 1 and 2, the lack of encapsulation will strangle our complexity as soon as we add any further complexity to our model, making further progress impossible.
- How do we determine relationships between entities?
- Who gets drawn in the background and who gets drawn in the foreground?
- When do we determine when we want to "share" a click event between overlapping elements, and when to stop sharing it?
We can tackle this in several models. Let's start with the simplest.
struct IEntity
{
virtual void update(float dt)=0;
virtual void draw(Canvas& ctx)=0;
virtual void handle(MouseEvent &m)=0; //test intersection & update state
}
This interface lets us create entities which will accept the canvas, an abstraction for the screen, and draw themselves. It will similarly accept a mouse-event abstraction to perform the "is user clicking and intersecting with me" test. Finally, update() lets an entity update their state, if any. We can manage our scene by keeping pointers to all subclasses of IEntity in a simple, fast container like a std::array or std::vector.
std::vector<IEntity*> Entities;
Given these entities, our render-loop functions now become.
void poll()
{
MouseEvent mouseEvt;
if (!pollMouse(mouseEvt))
return;
for (Entity* e : Entities){
e->handle(mouseEvt);
}
}
void update(dt)
{
for (Entity* e : Entities){
e->update(dt);
}
}
void draw()
{
Canvas& ctx = getGraphicsContext();
for (Entity* : Entities){
e->draw(ctx);
}
}
So this is much nicer, while getting the same functionality as above. However, we still have the overlapping events problem; if two elements both intersect then they both handle the event.
We might decide simply that IEntity's handle() method should return a "true" if we detect a hit "false" if it doesn't. Let's do this by changing its signature to returning a boolean and assume true means "hit". This is essentially the chain-of-command design pattern but with only a draw-state and side-effects being passed.
struct IEntity{
public void update(float dt)=0;
public void draw(Canvas& ctx)=0;
//now return true if we handle/don't want others to handle
public bool handle(MouseEvent &m)=0;
}
void poll()
{
MouseEvent mouseEvt;
if (!pollMouse(mouseEvt))
return;
//handle events in reverse of drawing order
for (auto e = Entities.rbegin(); e.!= Entities.rend(); ++e){
if (e->handle(mouseEvt))
break;
}
}
void update(dt)
{
for (Entity* e : Entities){
e->update(dt);
}
}
void draw()
{
Canvas& ctx = getGraphicsContext(); //gets the buffer we draw on
for (Entity* e : Entities){
e->draw(ctx);
}
}
Now, this design is better, however: what if we have a background Entity that we want to listen to mouse events regardless of what another event wants. Imagine it could be a giant eyeball keeping track of the mouse in a grisly fashion, or more simply just wanting the user to "grab" multiple items at once. What happens in this case? Do all of the prior entities return false? How do they know there is a background element waiting for input. Or, do we put some kind of special global listener check during the handling loop to take care of this special case? Ultimately, we would have to implement another hack to work around it. In such a case this approach unfortunately fails; fix a problem, get a new one.
----
We can't go back to the old approach because the issue of anarchy and entities being unable to communicate with one-another returns. Is there a better way?
We have to consider exactly how we want mutual interactions to work. There is a slightly more complex method that combines both approaches:
- every item will get a handle to an whatever event occurs
- there can only be one "active item"
- elements do not act if they are not the active item
The logic is in the following state-machine:
Entities can consider these global states if they want to, or they can ignore it if they want to act on the user's mouse input regardless of the other elements being active. You can enforce the state-machine's behavior with an API instead of using a dumb struct like I do. This approach lets us have conditional behavior fairly easily e.g. the background can decide to act only if the item currently being interacted with is the cockroach, but it is technically not necessary to do so. I would recommend giving Entity's unique id's here to distinguish one another.
// a struct to keep track of the current "active" object
struct HandlerState {
int active_elem;
} g_HandlerState;
struct IEntity{
virtual void update(float dt)=0;
virtual void draw(Canvas& ctx)=0;
virtual void handle(MouseEvent &m, HandlerState& hs)=0;
}
void poll()
{
MouseEvent mouseEvt;
if (!pollMouse(mouseEvt))
return;
//using -1 to represent "unassigned", any handle() method
//can take its state
g_UIState.active_elem = -1;
for (auto e = Entities.rbegin(); e.!= Entities.rend(); ++e){
e->handle(mouseEvt, g_UIState)
}
}
void update(dt)
{
for (Entity* e : Entities){
e->update(dt);
}
}
void draw()
{
Canvas& ctx = getGraphicsContext();
for (Entity* e : Entities){
e->draw(ctx);
}
}
This a much more canonical chain of command, and we can also view our state struct as a mediator.
Now here's the cool part and the thesis of this section: We can embed this same handling structure in a group of entities by creating one that is a composite of others.
struct CompositeEntity : IEntity
{
std::vector<IEntity*> subEntities;
public void update(float dt)
{
for (IEntity* e : subEntities)
e->update(dt);
}
public void draw(Canvas& ctx)
{
for (IEntity* e : subEntities)
e->draw(ctx);
}
public void handle(MouseEvent &m, HandlerState& hs)
{
for (auto e = Entities.rbegin(); e.!= Entities.rend(); ++e){
e->handle(mouseEvt, g_UIState)
}
}
We can do a lot of cool things on top of this structure such as partitioning the screen into different sections, such as subwindows, without the IEntity's inside being aware.
The overall structure we've built is pretty extensible; for example, we can sort our entity vectors based on some "graphical priority" where higher priority means drawing over/after low-priority IEntity's. We also can make our asynchronous by only redrawing when necessary or only subsets of the screen that need to be redrawn. We could even give our HandlerState object getters and setters to enforce our desired state-machine behavior!
Overall, I think a structure like this works best for event handling. Now we can move onto actual GUI design.
GUI
Finally, we're at the actual "GUI" part of this post. Now that we've pondered the fundamental idea of pushing events to entities, we can consider the primary models of GUI input handling and their relation to all of the above models.
Contemporary Models
The following are the two most common GUI architectures.
Retained Mode
- GUI Elements follow an object-oriented hierarchy in order to communicate the appropriate event to their child elements and furnishes default behavior
- Applications: Large OO Frameworks, OS APIs
- Examples: swing, win32, UWP/WPF .net
- GUI Elements are state-less, and the code implementing rendering is also implementing the logic; one function = one widget
- Applications: Games, Websites
- Examples: IMGui, Unity, Blender
Whatever GUI system we want, we ultimately have to implement it on-top of our real-time render-loop.
Implementing an Immediate Mode GUI
Immediate mode rendering is sort of like a short-cut. Instead of deferring the handing of input to some stateful object in your input queue, your do both simultaneously and return the result. Here's an example with the popular IMGui library.
while(true)
ImGui::Text("Hello, world %d", 123);
if (ImGui::Button("Save"))
{
// button has been clicked on
}
ImGui::InputText("string", buf, IM_ARRAYSIZE(buf)); //"buf" is pass-by-reference
ImGui::SliderFloat("float", &f, 0.0f, 1.0f); //uses ptr to the value being modified
This code would need to be executed sequentially within the same frame in the program loop. In terms of actual implementation, it looks a little more like this:
void draw(){
//...same prior stuff...
//hovering over mouse? (i.e. should it be hot)
if (mouseIsOverButton(mouseEvt) && mouseDown(mouseEvt){
drawButtonPurp();
doButtonAction();
}if (mouseIsOverButton(mouseEvt)){
drawButtonRed();
}else
drawButtonBlue();
}
}
"Hey now, that's crazy" you think, and you're mostly right. It's bad to mix logic and presentation code and it's is crazy to have to work directly within the context of the render-loop. This is why the bulk of IMGui's work is to hide these dark implementation details from you (by doing stuff like recognizing the rendering framework you're using). However, the main benefit here is that you know the extent and scope of all your GUI buttons and events. If you work within the event-loop this is very convenient. There is no chain of command or "leaving it up to the implementation" to decide what your layout will be. Instead the implementation is the layout, is also the logic.
Here's an immediate-mode program and. Its model harkens all the way back to the pre-entity render loop:
int r=0;
int g=0;
int b=0;
//draw all entities
void draw(sf::RenderWindow &w)
{
//input state-machine has to be prepared
imgui_prepare();
drawrect(100,100,150,50,sf::Color::Green,w);
//GEN_ID is a macro that generates a unique ID, which distinguishes the functions since
//drawing the button is technically a "state-less" operation
button(GEN_ID,100,200,w);
button(GEN_ID,100,300,w);
if (button(GEN_ID,200,150,w))
{
bgColor = Color(0,0,0); //black
}
//new value from slider?
if (slider(GEN_ID, 300, 100, 255, r, w))
{
bgColor = Color(r,g,b);
}
if (slider(GEN_ID, 400, 100, 255, g, w))
{
bgColor = Color(r,g,b);
}
if (slider(GEN_ID, 500, 100, 255, b, w))
{
bgColor = Color(r,g,b);
}
imgui_finish();
}
Believe it or not, since these aren't entities, our handle() function doesn't actually process() any input. These drawing functions themselves test our mouse state (which in this case is global), and then appropriately decide their actions.Immediate-mode GUIs, similar to our initial event-mode, use a simple scheme to resolve conflicts between multiple elements, which basically allows our simultaneous drawing/input-check scheme to work. Our fundamental assumption is the following: there can only be one GUI item working at one time. This is the active item.
If there is already an active GUI item, then that has claimed priority this frame. Furthermore, a GUI item must be hot before it becomes active. An item becomes hot by having user indicate their attention towards it. This is usually just done by mousing over an element before clicking it, but the hot item can also be switched by other mechanisms (e.g. tabbing often changes the input context for keyboard-driven programs). An item is only active as it is being acted upon (i.e. receiving input such as keystrokes or mouse-movement while holding down the left-mouse-button). As soon this input stops, the role of the active element is releived and other elements can now become active. Here's a diagram to tl;dr this:
So, let's put this into a full-code example with all the gory immediate-mode details. Here we'll implement a button and slider within the context of our poll $\rightarrow$ update $\rightarrow$ render loop.
//Immediate-mode button widget
bool button(int id, int x, int y, sf::RenderWindow& c)
{
//hovering over mouse? (i.e. should it be hot)
if (mouseIntersects(x,y,64,48))
{
g_mouseState.hotitem = id;
//should it be active?
if (g_mouseState.activeitem == 0 && g_mouseState.lmousedown)
g_mouseState.activeitem = id;
}
// -- keybd section --
//if no widget has keyboard focus, take it
if (g_mouseState.kbditem == 0)
g_mouseState.kbditem = id;
//if we have keyboard focus, show it
if (g_mouseState.kbditem == id)
drawrect(x-6,y-6,84,68,sf::Color::Red, c);
//render button
drawrect(x+8,y+8,64,48, sf::Color::Blue, c);
//this is the key state machine
if (g_mouseState.hotitem == id)
{
if (g_mouseState.activeitem == id)
{
drawrect(x+2,y+2, 64, 48, sf::Color::Cyan, c);
} else { //button is merely "hot"
drawrect(x, y, 64, 48, sf::Color::Cyan, c);
}
}
else //button is not hot, but it may be active
{
drawrect(x, y, 64, 48, sf::Color::Green, c);
}
//finally, check if button has been triggered
if (g_mouseState.lmousedown &&
g_mouseState.hotitem == id &&
g_mouseState.activeitem == id)
return 1;
return 0;
}
//Immediate-mode slider widget
bool slider(int id, int x, int y, int max, int &value, sf::RenderWindow &c)
{
int ypos = ((256.0-16.0) * value) / max;
//check for hotness
if (mouseIntersects(x+8,y+8, 16, 255))
{
g_mouseState.hotitem = id;
if (g_mouseState.activeitem == 0 && g_mouseState.lmousedown)
g_mouseState.activeitem = id;
}
drawrect(x,y,32,256+16, sf::Color(70,70,70), c);
if (g_mouseState.activeitem == id || g_mouseState.hotitem == id)
{
drawrect(x+8,y+8+ypos, 16, 16, sf::Color::White, c);
}
else
{
drawrect(x+8, y+8+ypos, 16, 16, sf::Color(200,200,200), c);
}
if (g_mouseState.activeitem == id)
{
int pos = g_mouseState.y - (y+8);
if (pos < 0) pos=0;
if (pos > 255) pos = 255;
int v = (pos * max)/255;
if (v != value)
{
value = v;
return true;
}
}
return false;
}
void imgui_prepare()
q{
g_mouseState.hotitem = 0;
}
void imgui_finish()
{
if ( !g_mouseState.lmousedown ){
g_mouseState.activeitem = 0;
}
else
{
if (g_mouseState.activeitem == 0)
g_mouseState.activeitem = -1;
}
}
//bg set-up
sf::Color bgColor = sf::Color::White;
int r=0;
int g=0;
int b=0;
//draw all entities
void draw(sf::RenderWindow &w){
//resets active element if mouse-button is up
imgui_prepare();
drawrect(100,100,150,50,sf::Color::Green,w);
//this macro generates a unique id for each element
button(GEN_ID,100,200,w);
button(GEN_ID,100,300,w);
if (button(GEN_ID,200,150,w)){
bgColor = sf::Color::Black;
}
if (slider(GEN_ID, 300, 100, 255, r, w)){
bgColor = sf::Color(r,g,b);
}
if (slider(GEN_ID, 400, 100, 255, g, w)){
bgColor = sf::Color(r,g,b);
}
if (slider(GEN_ID, 500, 100, 255, b, w)){
bgColor = sf::Color(r,g,b);
}
//resets active element for next frame
imgui_finish();
}
The output will look something like this.
When implementing this manually the main drawback is clear: our input handling is limited by our drawing order and state-machine. Embedding requires a bit more thought to implement. The scaffolding immediate most is the most basic of the basics, but it's also a doable start when building a new real-time system.
We can mitigate these drawbacks with some basic factoring approaches; on your own library you'll want to factor out as much behaviors into smaller state-less functions until you build a satisfactory compendium of widgets. For example, we can factor out the generation of unique ids for each element, and we can also maintain more sophisticated states in some type of GUI singleton. The entire trick to the immediate-gui approach is to keep the state-machine's invariants intact through each widget's draw() implementation.
Immediate-Mode GUI
Pros:
-Ease of use for low-level developers
-Simplicity of concept allows for easy debugging
-Immediately reactive, trivial to implement on-top of a real-time system
Cons:
-Not as scalable
-Coupling of model and presentation
-You have to be at a low-level for this to be useful
a full implementation of the above can be found here
----
Implementing a Retained-Mode GUI
So the main drawbacks of the previous interface were the implementation: there is essentially no inheritance in the immediate mode model. Rather, widget's implementations are more-or-less hard-coded and cannot be easily extended or embedded. Enter retained-mode.The basic goal of retained-mode GUI frameworks is to let the end-user think purely in terms of high-level abstractions. You don't have to think about the event-loop or what happens when a user mouses-over your widget, but it gives you the ability to delve further if you so wish. Retained mode provides sufficient defaults such that you can construct each sub-component independently without concerning yourself with how it all interacts as a whole. The Object-Oriented design takes care of the details of the interaction as it constructs the runtime of the program.
As an example, here's java swing's class hierarchy.
 | |
| JComponent is the base-class of all pre-made widgets, and each widget holds its state |
class GUIWidget;
class Window : IEntity
{
std::vector<GUIWidget*> children;
Layout guiLayout;
public:
void draw();
void handle(const MouseEvent &e);
void update(float dt);
}
Window::handle(const MouseEvent &e)
{
for (GUIWidget* gt : children)
{
bool wasHandled = false;
if (gt->handles(e))
try
{
gt->do_handle(e);
wasHandled = true;
}
catch (DelegationException ex)
{
this->popup(ex.what());
}
}
if (!wasHandled)
this->defaultHandle(e);
}
Furthermore, in most retained-mode API's, each widget can listen to events other widgets receive. Everything has an identifier: most frameworks let you can assign unique id's to a particular widget, but otherwise an element will be identified by a default id or relative path. This is where retained-mode starts offering a lot more flexibility but also becomes prone to clearly-bad-code-design. The nuclear option of retained-mode is for elements to recurse on their containers and analyze their siblings (and it happens more often than you think!). The inter-widget system can potentially share similar complexities to those in an IPC scenario: you start to make too many ways to accomplish the same operations and accumulate too much redundancy in the implementation.
On the bright-side a less obvious benefit of retained-mode's statefulness is how we can easily utilize it statefulness to serialize/deserialize our GUI objects. This can be extended as far as to create higher-level, human-readable file representations of our GUI system, such as what windows UWP platform does. The designer of our GUI form can be independent of the logician implementing its functionality. Here's an example with C# and UWP.
Definition of a page in UWP:
<Page
x:Class="agf_parser_uwp.SettingsPage"
xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
xmlns:local="using:agf_parser_uwp"
xmlns:d="http://schemas.microsoft.com/expression/blend/2008"
xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
mc:Ignorable="d">
<RelativePanel Background="{ThemeResource ApplicationPageBackgroundThemeBrush}">
<!--each of these things should have handlers like "SelectionChanged_thing"-->
<ListView HorizontalAlignment="Left"
VerticalAlignment="Top" Margin="25,25,0,0">
<ComboBox x:Name="FontSize"
Header="Font-Size"
HorizontalAlignment="Left"
SelectionChanged="FontSize_SelectionChanged">
<ComboBoxItem Content="Small" />
<ComboBoxItem Content="Medium" IsSelected="True"/>
<ComboBoxItem Content="Large" />
</ComboBox>
<ToggleSwitch x:Name="VaToggle"
Header="Voice Acting"
HorizontalAlignment="Left"
Height="60"
Width="154"
Toggled="VaToggle_Toggled"/>
<!-- launch URI for my site, more info for this here: https://docs.microsoft.com/en-us/windows/uwp/launch-resume/launch-default-app -->
<TextBlock Text="Written By Sergey Ivanov" />
</ListView>
</RelativePanel>
</Page>
Definition of handlers for the above page:
public sealed partial class SettingsPage : Page {
public SettingsPage()
{
this.InitializeComponent();
}
private void VaToggle_Toggled(object sender, RoutedEventArgs e)
{
ToggleSwitch o = e.OriginalSource as ToggleSwitch;
GlobSetting g = GlobSetting.getInstance();
g.text_speech = o.IsOn;
}
private void FontSize_SelectionChanged(object sender, SelectionChangedEventArgs e)
{
//do a thing
ComboBox o = sender as ComboBox;
ComboBoxItem n = o.SelectedItem as ComboBoxItem;
// do something with n.Content;
}
}
The final benefit I'll mention (but won't implement) is that retained mode's self-awareness can be applied to dynamic widget layout: since we have knowledge of all our child and sibling elements, we can automatically align ourselves on events like window resizing. Though this requires more possibly unnecessary work, a GUI made automatic for the end-user becomes a space partitioning problem for the implementer.
Implementation
A basic retained-type system in C++ is as follows:
struct GUIEntity : IEntity {
//new retained-mode stuff
std::string string_id_;
static GUIEntity* getElementById(std::string search_id_) {
//iterate through all types, and if it's a GUIEntity type that matches the id, then return that
GUIEntity *ret{nullptr};
for (IEntity* t : Entities) {
GUIEntity* gent = dynamic_cast<guientity>(t);
if (gent && gent->string_id_ == search_id_)
ret = gent;
}
return ret;
}
//useful globals
static int id_ctr_;
static UIState& uistate_;
static int active_id_;
//info all GUIEntities must have
int id_;
sf::Vector2i pos_;
sf::Vector2i size_;
GUIEntity(std::string strid="") //slightly modified c-tor
{
id_ = id_ctr_++;
string_id_ = strid;
}
bool intersect(sf::Vector2i pos)
{
if ( pos.x < pos_.x || pos.x >= pos_.x + size_.x ||
pos.y < pos_.y || pos.y >= pos_.y + size_.y )
return false;
return true;
}
};
int GUIEntity::id_ctr_ = 1;
UIState& GUIEntity::uistate_ = g_mouseState.ui;
struct Button : GUIEntity
{
std::function<void()> cb_;
void update(float) { /*do nothing*/ };
Button(int x, int y, std::function<void()> callback, std::string strid="")
: GUIEntity(strid), cb_(callback)
{
pos_ = {x,y};
size_ = {64,48};
};
Basically, each entity has a unique integer id, optional string id, and references to its variable states. In this case the string id can be searched with a standard c++ find, though a more industrious system might demand a hashmap and id uniqueness. We put this interface in a static method for convenience.
Entities would be used as shown below. Note how we can use our getElementById method to get other elements in the scene.
//rgb sliders, first one doesn't actually trigger the BG color value's update
Entities.push_back(
new Slider(300,100,0,255,r, [](int){},"red")
);
Entities.push_back(
new Slider(400,100,0,255,g,[&](int){bgColor = sf::Color(r,g,b);}, "green")
);
Entities.push_back(
new Slider(500,100,0,255,b,[&](int){bgColor = sf::Color(r,g,b);}, "blue")
);
//now we give this button functionality
Entities.push_back(
new Button(100,200, [](){
Slider *rslider = dynamic_cast<slider>(GUIEntity::getElementById("red"));
Slider *gslider = dynamic_cast<slider>(GUIEntity::getElementById("green"));
rslider->ref_ = 120;
gslider->ref_ = 120;
gslider->cb_(120);
})
);
a full implementation of the above code can be found here.
The main cons are all the non-functional scaffolding you have to develop and maintain. Perhaps the biggest con is the flip-coin of the all the pros: because you're deferring so much behavior to default, and so much state-control to the implementation, you don't have as much ability to determine how your widgets react to non-trivial events or how it looks; it's a bit trickier to define "if the user hits escape after mousing over the widget 3 times the widget flashes randomly" in a retained-mode than immediate-mode, and indeed this is because of the use-case retained mode has assumed for you. By providing a convenient scaffolding to make forms easily and quicky, it has also boxed you into that same framework, thus the flaws of a poorly implemented retained-mode GUI will be much more apparent than the flaws of a poorly implemented immediate-mode GUI.
Still, the advantages are so overwhelming that the majority of windowing toolkits use the retained-model over the immediate-model.
Pros:
-Extreme flexibility
-Objects can factor out a lot of common functionality
-Extreme convenience to end-user
Cons:
-There is a lot of non-gui work to do, annoying to implement
-Automatic layout and other reflection-based activities are more difficult
-Immediate-mode is a quicker GUI that puts more burden on the end-user, but is easier for game projects like GnomeAdventure.
----
My Preferred GUI Architecture So Far
If we combine this GUI architecture analysis with our object model in the first part of this post, then we get to my current preferred GUI method, which is basically a more sophisticated immediate mode.I figured I wanted an immediate-mode I could build on that could be easily extended to support window overlapping, element-id's (the retained-mode mode code is an extension of this code), and a clean interface to the state machine that isn't explicitly tangled with the main handling loop.
struct IEntity
{
virtual void update(float dt)=0;
virtual void draw(sf::RenderWindow& r)=0;
virtual void handle(MEvt& mevt)=0;
};
struct GUIEntity : IEntity {
//useful globals
static int id_ctr_;
static UIState& uistate_;
static int active_id_;
//info all GUIEntities must have
int id_;
sf::Vector2i pos_;
sf::Vector2i size_;
GUIEntity()
{
id_ = id_ctr_++;
}
bool intersect(sf::Vector2i pos)
{
if ( pos.x < pos_.x || pos.x >= pos_.x + size_.x ||
pos.y < pos_.y || pos.y >= pos_.y + size_.y )
return false;
return true;
}
};
int GUIEntity::id_ctr_ = 1;
UIState& GUIEntity::uistate_ = g_mouseState.ui;
struct Button : GUIEntity
{
std::function<void()> cb_;
void update(float) { /*do nothing*/ };
Button(int x, int y, std::function<void()> callback) : cb_(callback)
{
pos_ = {x,y};
size_ = {64,48};
};
void draw(sf::RenderWindow& r)
{
if (uistate_.activeitem == id_) //being clicked
{
drawrect(pos_.x+2, pos_.y+2, size_.x, size_.y, sf::Color::Cyan, r);
cb_();
}
else if (uistate_.hotitem == id_ && uistate_.activeitem != id_)
{
drawrect(pos_.x, pos_.y, size_.x, size_.y, sf::Color::Cyan, r);
}
else //I am neither hot nor active
{
drawrect(pos_.x, pos_.y, size_.x, size_.y, sf::Color::Green, r);
}
}
void handle (MEvt& mevt)
{
if ( intersect({mevt.x,mevt.y}) )
{
if (uistate_.hotitem == 0)
uistate_.hotitem = id_;
if (mevt.lmousedown && uistate_.activeitem == 0)
uistate_.activeitem = id_;
}
}
};
struct Slider : GUIEntity {
int min_;
int max_;
int &ref_;
std::function<void(int)> cb_; //optional callback
Slider(int x, int y, int min, int max, int &value, std::function<void(int)> cb=nullptr) :
min_(min), max_(max), ref_(value), cb_(cb)
{
pos_ = {x,y};
size_ = {32,256};
}
void update(float) { /*do nothing*/ }
void handle(MEvt& mevt)
{
if ( intersect({mevt.x,mevt.y}) )
{
if (uistate_.hotitem == 0)
uistate_.hotitem = id_;
if (mevt.lmousedown && uistate_.activeitem == 0){
uistate_.activeitem = id_;
}
}
//while I am the active item
if (uistate_.activeitem == id_){
//do slider action
int pos = mevt.y - (pos_.y+8);
if (pos < 0) pos = 0;
if (pos >= size_.y) pos = size_.y-1;
int v = (max_-min_)*(pos/255.0) + min_;
if (v != ref_){
ref_ = v;
if (cb_ != nullptr){ cb_(ref_); };
}
}
}
void draw(sf::RenderWindow& rw)
{
//main bar always gets drawn the same way
drawrect(pos_.x, pos_.y, size_.x, size_.y + size_.x/2, sf::Color(70,70,70), rw);
int sliderDrawPos = ((ref_ - min_) / (float)(max_ - min_))*size_.y + (pos_.y + size_.x/4); //recompute slider's pos from its referenced val
if (uistate_.hotitem == id_ && uistate_.activeitem == id_)
drawrect(pos_.x + size_.x/4, sliderDrawPos, size_.x/2, size_.x/2, sf::Color::White, rw);
else
drawrect(pos_.x + size_.x/4, sliderDrawPos, size_.x/2, size_.x/2, sf::Color(200,200,200), rw);
}
};
The clever part is the reference to the GUI state, which itself is technically global as it gets updated by the start of each frame, but the local interface provides the correct abstraction with the appropriate side-effects without having to sloppily just pass a global around.
This approach scales well to a 3D model, as GUIs are essentially organized entities, and an embedded GUI in some texture or other real-time element just becomes the same hierarchy but conditioned on standard 3d intersection with it.
a full implementation of the above can be found here
Conclusion
The best models work because of a separation of concerns between abstractions and leveraging a few useful design patterns.
When considering an implementation, retained system has obvious benefits. In-fact, it's what I went first with because of how comfortable it is to just define a specific handler or subset of an already preconcieved form. However for a game or other new system where you're designing the GUI from scratch, you are generally better served by building your GUI with an immediate model, even in GUIs of moderate complexity. The best approach is to build up from an immediate GUI in a way that you can transition to a retained architecture if necessity demands it.
Perhaps the biggest takeaway here, is the IMGUIs and Retained GUIs are two-sides of the same coin. Fundamentally, there must be an "immediate" layer to all rendering systems, even asynchronous ones, and anything above that is someone's clever ideas.
No comments :
Post a Comment