Dev - UI - Accessibility
Tools
Discussion on HN https://news.ycombinator.com/item?id=27287270
ClawsOnPaws 5 hours ago [–]
I know that this project is still in very early stages, however as I often do, I’ve come here to mention accessibility. I’ve run the examples on both Mac and Windows, and as expected, I can’t use either of them. This is not a strict criticism yet and I’m aware how difficult it is to interface with accessibility API’s, but I do hope that if this project does make it far enough that this will be considered. Pretty much everything that draws their UI directly to the screen using OpenGL or similar immediately makes me think that I probably don’t even need to try the program to know that it’s inaccessible, and more often than not I’m right. If you’re building an app using these gui toolkits, please do keep this in mind. reply
mtzet 49 minutes ago [–]
This gets brought up pretty much every time immediate mode GUIs are discussed on HN. I think your comment is reasonable, and I don’t disagree, but I also think it’s important to not throw the baby out with the bathwater and dismiss immediate mode GUIs for accessibility reasons. It’s not at all clear to me that the right approach is to have both the accessibility interface and the GUI be generated from the same components. It seems to me that it’s just approaching the lowest common denominator of a passable GUI and a passable accessibility interface.
The traditional widget-style GUI paradigm asks you to essentially copy all your application data into their format and to keep this format in sync with your own data. This is tedious, and I believe hinders the creation of better GUIs. On the flip-side, owning your data structure means it can indeed hook into the accessibility API for you.
Immediate mode GUIs provide an alternative approach, where they only handle rendering stuff on screen. Since all the data structures are now properly controlled by the programmer, it is much easier to create dynamic GUIs. Taking a step back like this provides an opportunity to further the state of the art.
The solution I see to providing accessibility in immediate mode GUIs is that the programmer has to hook into the accessibility API themselves — if it makes sense. It’s not clear to me that certain advanced GUI programs can ever be very accessible. Yes, this is more work, but there’s also an opportunity to create richer experiences for people with disabilities.
reply
pqb 3 hours ago [–]
This, hundreds times this. I can’t say how many years in a row I was experimenting with various GUI toolkits, libraries and concepts of an application like rxi/lite editor 0. While I saw many good looking implementations but the very first what caught my attention was lack of any kind support to screen-reading software or problem with displaying right-click context menus. I even remember previous HN discussion when I pointed out few projects that could provide some screen-readers support 1. Sadly, I would say 95% of all GUIs I know do not care about the accessibility. From the positive side, I am glad more and more hackers on HN focus on that matter. Also worth noting big players on the market really do take that point seriously, e.g. Flutter has a Semantics tag 2.
reply
slimsag 2 hours ago [–]
If we want more accessible UIs, we need to demand better standard accessibility APIs from OS vendors Microsoft and Apple. Unity has no official support for screenreaders or colorblind modes. Unreal has both. If an major game engine developer like Unity cannot be bothered to add support, can we really expect the little developer hacking away on some OpenGL side project to?
Most people working with OpenGL use GLFW (or sometimes SDL), neither have any cross-platform API for supporting screen readers. Why? Because not only do Windows and MacOS have different APIs, but different screen readers, braille displays, etc. have different APIs.
But okay - let’s look at the web. It’s the most accessible platform of them all, right? What have Google/Apple/Mozilla done to support developers adding screenreader support from WebGL and Canvas-based applications? → not a whole lot. You need to inject text into a hidden div, which also has performance implications so you need to build a UI to toggle it on/off or detect accessibility the way Flutter Web does.
We should have better accessibility, but as long as we expect developers to run through several hours/days of hoops to get even something basic working - it’s just not going to happen, and that is mostly the fault of extremely poor platform APIs.
reply
mwcampbell 46 minutes ago [–]
extremely poor platform APIs As a former member of the Windows accessibility team at Microsoft, I’d appreciate your thoughts on what makes the platform accessibility APIs extremely poor. I have my own thoughts on what makes them difficult to implement, but I’d like to hear your perspective first.
reply
iudqnolq 1 hour ago [–]
Unity has no official support for screenreaders or colorblind modes. Unreal has both. If an major game engine developer like Unity cannot be bothered to add support, can we really expect the little developer hacking away on some OpenGL side project to? Games are special because the means of interaction is often a core part of the experience. An accessibility mode for a game can be a huge amount of work. Not to say that some people don’t do that huge amount of work (which is great) but games lacking accessibility is no excuse to use game-like development practices to write apps that could otherwise be relatively easily made accessibile.
reply
amelius 2 hours ago [–]
In my opinion, accessibility should be implemented at the OS level. The OS can, for example, run OCR on the entire screen and turn bitmaps into selectable text, read text out loud, etc. This kind of functionality shouldn’t be replicated on a per-app basis. Accessibility of browsers is usually considered to be very good, but browsers are almost OSes, so why not lift this one more level into the OS? reply
ClumsyPilot 4 hours ago [–]
This is the one real (and legal) reason why people should actually use OS UI. To replicate this functionality is a tall order. The ‘looks native’ thing is mostly noise
reply
Ennea 3 hours ago [–]
As someone who occasionally writes tiny tools with some GUI, how can I test how well they work with accessibility tools? reply
supercheetah 2 hours ago [–]
On Windows, test it with NVDA1 or JAWS2. On Linux, in Gnome, there’s a screen reader that can be enabled in the settings under accessibility. I’m not sure what there is for Mac, Android, or iOS offhand. Also, test in high contrast modes, and screen magnifiers.
reply
mwcampbell 13 minutes ago [–]
To expand on this: Windows also has the Narrator screen reader built in, and in Windows 10, Narrator is a decent screen reader IMO. (Disclosure: I used to be on the Windows accessibility team at Microsoft, where I worked on Narrator.) To enable Narrator, press Ctrl+Win+Enter. Narrator has a built-in tutorial (not written by me) to help you get started.
Mac and iOS has VoiceOver built in. To enable it on Mac, press Command+F5. On iOS, you may be able to triple-press the home button, on devices that have one. Failing that, you can find it in Settings.
Android has TalkBack built in. You can find it somewhere in Settings; the specifics are device-dependent.
reply
Ennea 1 hour ago [–]
Thank you, that’s very helpful! I’ll be saving these for later :) reply
gjvnq 19 minutes ago [–]
Hacky/Unixy solution: Make a CLI version. reply
tapirl 3 hours ago [–]
It is very helpful for the developers if you could provide more info, such as error messages. reply
iamgopal 2 hours ago [–]
Ultimately browser as a gui, ( even without JavaScript ) will win, no other gui efforts are at par with that. So a WebKit with api to plug any language backend would be a good idea ( electron without js ). Is there any effort in that direction ? reply
vincnetas 6 hours ago [–]
Im gonna be grumpy old man. It’s nice to play with new things, but it’s really sad when UI conventions from OS is not implemented. This makes using this UI very frustrating: Triple click on the text. Expected whole paragraph to be selected.
In input field click Cmd+A to select all text. Does not work as expected on mac.
Copy paste does not work.
Start selecting text but release mouse outside of the widget. Text selecting gets stuck.
Text navigation in text field does not work Cmd - left/right.
And these are from 5 minutes of playing with it.
ps: im on mac+safari
reply
rob74 4 hours ago [–]
Ok, let’s try to say “the glass is already 20% full, not 80% empty” - there are still lots of things missing, but hey, it’s still early days… My pet peeve with most “home-grown” GUI frameworks is relatively subtle and probably hard to implement, but it’s an example why developing a GUI framework (especially a platform-independent one) from scratch is hard: no subpixel rendering for text (only “classic” antialiasing). This makes these UIs look strange on a “normal-DPI” monitor (which are getting rarer these days, but I still have some and I’m not gonna throw them away anytime soon).
reply
swiley 1 hour ago [–]
Something a lot of people re implementing GUI toolkits on OSX might not realize is that emacs/readline shortcuts work in all of the textfields (since ctrl is more or less “free” thanks to the command key.) IMO the GUI toolkit is one of those things OSX got really right and if you’re not going to bother using it you might as well just target X11 and save yourself the trouble of putting effort into your OSX port. reply
amarshall 1 hour ago [–]
The bindings are also customizable in ~/Library/KeyBindings/DefaultKeyBinding.dict.
reply
buffrr 5 hours ago [–]
I have been following Gio development for quite some time, and it’s still in the early stages. Text selection was added recently. It could be better but it’s a good start. Double click to select a word and triple click to select paragraph doesn’t work for me either but can’t reproduce some of the other issues running it natively.
reply
daenney 5 hours ago [–]
I think this is more of an artifact of the browser/WebGL things still being experimental. I don’t observe broken behaviours around copy paste or text selection if I run the example apps natively. reply
nyanpasu64 5 hours ago [–]
I can’t right-click a static/input text selection and get a menu (whether a browser menu or a non-native menu) with the ability to copy and paste and such. reply
user-the-name 4 hours ago [–]
Also, no accessibility. macOS has amazing support for making its UI accessible to a wider variety of people, but immediate mode UIs throw this all out the window. reply
ktpsns 5 hours ago [–]
Immediate mode GUIs seem to be some “fresh wind” to desktop GUI toolkits. Apparently they are heavily motivated by gaming and 3D focussed applications. I wonder whether they are also play a role in traditional desktops. The Gio kitchen sink example counts hundred lines of code (https://git.sr.ht/~eliasnaur/gio-example/tree/main/kitchen/k…), but for instance Dear ImGui examples (https://github.com/ocornut/imgui) are suprisingly short and require no headache with layouting containers. I could imagine that coding GUIs from the scratch is “fun again” with such a toolkit. Has anybody carried out a larger traditional (i.e. non 3D focussed) application with an immediate mode GUI toolkit?
reply
antattack 7 minutes ago [–]
Immediate mode GUIs seem easier to build and parse but harder to make changes to/maintain. Perhaps this is why they are popular in games. I am patiently waiting for release of fidget2, immediate mode GUI Nim library myself. https://github.com/treeform/fidget
reply
whizzter 5 hours ago [–]
Not that I know of, the issue is often that most of them run through graphics API bindings currently. In many ways immediate mode UI’s is a relative of declarative UI’s like React and I think we might see hybrids or implementations of either on top of the other.
Why? Right now immediate mode UI’s often lack accessibility,etc and since it’s often game/graphics focused there has been little push to fix this afaik. But there really is little reason not to implement an im-gui that renders to a diff-able tree that then is translated to “Native UI components”.
Imo one big reason why declarative UI’s hasn’t caught on is because C/C++ really don’t have the best ergonomics for building “flexible” data structures or immutable updates, by turning it around to using im-gui style rendering it could probably be solved fairly smoothly.
reply
flohofwoe 5 hours ago [–]
Check out the Dear ImGui screenshot threads, there are quite a few examples of applications with complex UI in there: https://github.com/ocornut/imgui/issues/3488
reply
rob74 6 hours ago [–]
I’ve been following the evolution of GUI bindings/native solutions in Go for a while now, and unfortunately all attempts at “Go-native” GUIs like this one have stalled long before they could achieve even a small fraction of the functionality of “established” frameworks like Qt or GTK (although I’m not sure that’s even a goal for this project). That said, I wish them (and the other projects which are still active) good luck… reply
jonathanstrange 4 hours ago [–]
The Fyne developers are still very active. I don’t think it’s useful for more than small utilities in its current state, but I hope they’ll implement some well-working multiline rich text editing soon. This is the area in which most homebrewn GUIs tend to fail. Rich text editing is surprisingly hard to get right and reasonably platform-compliant. reply
daenney 5 hours ago [–]
It’s definitely a hard thing. Gio is still actively developed and it can already do useful things. On the not 100% native side there’s Fyne which seems to only depend on xorg development headers. It can do a lot.
reply
psanford 54 minutes ago [–]
Gio development isn’t stalled. reply
sigzero 27 minutes ago [–]
He didn’t say it was. He was talking about “in general” and at the end even wishes them good luck. reply
Zababa 3 hours ago [–]
Are there any more resources about immediate-mode GUI compared to retained-mode? I’ve read https://eliasnaur.com/blog/immediate-mode-gui-programming and I’m not sure I get it exactly. Is this because retained-mode toolkits don’t offer enough control to developers so they want something lower level? reply
jay_kyburz 6 hours ago [–]
Do you think the UI is supposed to fly off and spin around in space when you click on it? https://imgur.com/a/sDmUA2x
reply
jay_kyburz 6 hours ago [–]
Oh, it happens when you hit the transform checkbox. reply
pansa2 6 hours ago [–]
It’s good to see an example of Go running in the browser via WebAssembly. I’ve been trying to decide between Go and C++ for targeting WASM, but it’s not clear whether Go is a good choice for this and whether WASM support is a priority for the language’s developers. reply
gher-shyu3i 5 hours ago [–]
golang lacks the language facilities to implement GUIs in a concise manner. This is one place where inheritance makes things a lot easier. reply
ph 5 hours ago [–]
I don’t think you need inheritance for this at all. Inheritance might make providing an API a little bit easier, but it is not required, especially not for the consumer of an API. That Go structs have methods and you can embed structs into other structs and also be able to call those methods gives just enough “object orientation” which is necessary. The Go gtk bindings are a good example for this. For the user, they feel like a normal object oriented API. reply
gher-shyu3i 5 hours ago [–]
Strictly speaking, no language features are required other than the bare minimum, and we can write everything in assembly. The fact of the matter is that golang remains has failed to make headway into the UI space, and its language features make it very poor to work with for large code bases. reply
ph 4 hours ago [–]
That is a non-argument. I was clearly not speaking about the bare minimum, but about required in a practical sense. Have you looked at the Go gtk bindings? If not, do so! The Go bindings are way nicer than the original C interface, Gtk was designed for. Gtk fundamentally has an object-oriented design. As it is based on pure C, it has its own thin object layer. However the “object oriented” features of Go blend much better with that design than the native C. Because you can build type hierarchies of structures and each structure type has methods. I very much regret that Go hasn’t made bigger headway into the UI space, but it is a chicken and and egg problem. You need UI bindings to write UI applications. In general, there isn’t enough development happening, especially for Linux. And alas, too many new developments get lured into writing electron apps. And there are of course a lot of QT apps. But in my eyes, Go+Gtk should be a great combination for the desktop, as the bindings have a nice API and Go is really approachable for newcomers - it was one of the design criteria to not have too high learning efforts. If anything, there could be more documentation on how to use Gtk. And so far, my Go+Gtk programs can be easily recompiled and run on the Mac :)
reply
jonathanstrange 4 hours ago [–]
I agreed with you when you were talking about inheritance, but the claim that Go is not good for working with large code bases seems odd to me. Working with large codebases was one of Go’s design goals. The idea is to split up programs into small packages with clear APIs, and these can be developed and maintained independently by hundreds of developers. I’d say it works quite well. (I’m not claiming it’s the best, just that it works well. Technically, I’d still prefer Ada’s extreme modularity with explicit interface definitions.)
reply
gher-shyu3i 52 minutes ago [–]
Working with large codebases was one of Go’s design goals. Which they failed to meet. There is nothing special in golang that is designed for large codebases. The opposite in fact, things like how they designed interfaces is counter to large code bases. There are much better languages that do well in large code bases.
reply
greyman 4 hours ago [–]
has failed to make headway into the UI space, That’s true (so far).
and its language features make it very poor to work with for large code bases.
Not in my experience. We have quite a large codebase at work and we implement new features just fine. At least it looks better than in my previous C++ projects, where after a few years the code structure became a big rigid.
reply
gher-shyu3i 51 minutes ago [–]
C++ is not really the benchmark to try to beat. There are superior languages for large code bases, but they’re not “hip”. reply
pjmlp 4 hours ago [–]
It only failed because no one was willing to spend the money it requires to make it happen. See how much money might already been spent into Flutter development.
reply
pjmlp 4 hours ago [–]
COM/UWP doesn’t do class inheritance, only interfaces. It also provides delegation mechanisms. Just like Win32 object model, is based on message dispatch across windows classes, not necessarly inheritance.
Just like game engines based on component architectures.
reply
pwdisswordfish8 3 hours ago [–]
Should I even bother mentioning GIO, an I/O library closely tied to Gtk, given that nobody cares about name clashes anymore? https://developer.gnome.org/gio/2.68/
reply
Zababa 3 hours ago [–]
Is there a tool that allows for checking such name clashes, ideally by specifying the domain to see how close you are? reply
zserge 1 hour ago [–]
You mean like a web search engine? A quick search on “gio lib” shows a wiki for Gtk’s GIO as the first result. You might narrow down the search adding domain-specific keywords to it, like “gio lib golang” to find something like this - https://github.com/vlorc/gioc (completely different field and name, but still can be found). reply
Zababa 1 hour ago [–]
In that particular case a web search works, sure. But for most name clashes that I see, it’s often tools that are well known in some domains but not so much in others. reply
Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact
Search: