keyboard layout evaluation
there are a number of keyboard layout analysers, some web based, some desktop applications. The readily accessible web based analysers use a graphical interface to input a new layout for comparison against a predefined library of stock layouts—QWERTY, Dvorak and Colemak are sure to be amongst them.
Each analyser’s scoring algorithm reflects the programmer’s approach to keyboard layout design, typically calculating finger distance traveled and applying finger weights and penalties (for finger, same finger, row and finger roll direction) to yield a score against a given corpus. Some provide a scoring breakdown, such as, finger distance traveled, alternating/same hand usage, inward/outward rolls, etc.
The variety of physical keyboards available with their column slant and/or row stagger, not to mention layers, split keyboards and tenting solutions, reveals a large problem domain—all of which has garnered little academic study. So it has been up to keyboard enthusiasts and programmers to tackle the problem. Some algorithms are quite complex, others more rudimentary in their assumptions. Each reflects the designer’s goals and preferences e.g. alternating hand usage, minimal row jumps, finger preferences, etc.
None to my knowledge can handle layers as created by QMK firmware or its many keycode functions—not to mention the customized functionality one may inject to enhance writing. Regardless, the scores calculated can provide a meaningful measure for comparing the base layer of differing keyboard layouts.
On the road to BEAKL Wi i have used a variety of web based analysers, most rather crudely, treating the scores as indicators and measures with which to compare my own layout variants against each other. Of course, it’s always gratifying to glimpse how one’s latest layout measures against the alternatives—and determine the potential benefit of other layout approaches. At the end of the day, though, the acid test is living with one’s changes and determining its worthiness, something a number can only hint at.
kb-layout-evaluation
i happened to come across the desktop kb-layout-evaluation application and liked what i saw for several reasons, not least of which, its simplicity to configure, modify and run. It even outputs a coloured graph of the results! But it is the straightforward editable configuration file (versus the common json file structure of web analysers) that is the attraction, being a die hard command-line interface user.
It is not as sophisticated as other analysers in some regards, does not handle layers and modifier keys, is limited to bigram analysis, and ignores the Space key (does not define it). That being said, it allows adjusting its weighting and penalty system, including finger roll direction. Control of these values permits tuning the goal of the analyser.
Not perhaps as accommodating or thorough as other analysers, omitting metrics such as finger travel, alternating hand and inward/outward finger rolls, etc., its focus is on evaluating typing words (not programming code) on ortholinear and split keyboards (as reflected by the symmetrical weighting scheme). As such, it meets my needs for evaluating incremental layout changes and comparison of those results.
thumbs
all that being said, kb-layout-evaluation cannot accommodate my BEAKL variants out of the box. The script.py analysis program does not handle thumb keys which would be required for including the Thumb I.
Adding 4 lines of code to script.py appears to address this limitation, 2 to define the thumb key..
def parse_config(blocks):
...
# assign a letter per key corresponding to which finger is used
for row in df_keys.itertuples():
if int(row.Index[2:]) >= 7: # -> thumb row <-
df_keys.at[row.Index, 'finger'] = 't' # -> thumb row <-
# pinky
elif int(row.Index[2:]) <= 2:
df_keys.at[row.Index, 'finger'] = 'p'
# ring
elif int(row.Index[2:]) == 3:
df_keys.at[row.Index, 'finger'] = 'r'
# middle
elif int(row.Index[2:]) == 4:
df_keys.at[row.Index, 'finger'] = 'm'
# index
elif int(row.Index[2:]) >= 5:
df_keys.at[row.Index, 'finger'] = 'i'
2 to handle the thumb key top row bigram reach..
def weight(bigram, layout, df_layouts, df_keys, df_penalties):
...
# penalty exists only if same hand, and not same letter
penalty = 0.0
if (key1[0] == key2[0]) and (bigram[0] != bigram[1]):
# define the row jump (column name in df_penalties)
if(abs(keyrow1 - keyrow2) == 0):
rowjump = "same_row"
elif(abs(keyrow1 - keyrow2) == 1):
rowjump = "row_jump1"
elif(abs(keyrow1 - keyrow2) == 2):
rowjump = "row_jump2"
elif(abs(keyrow1 - keyrow2) == 3): # -> thumb row <-
rowjump = "row_jump3" # -> thumb row <-
else:
sys.exit('Penalty for line jump not defined')
The config.txt file is updated accordingly for the additional thumb row 5 and its corresponding weights..
[keys]
L21 L22 L23 L24 L25 L26 R26 R25 R24 R23 R22 R21
L31 L32 L33 L34 L35 L36 R36 R35 R34 R33 R32 R31
L41 L42 L43 L44 L45 L46 R46 R45 R44 R43 R42 R41
L51 L52 L53 L57 L58 L59 R59 R58 R57 R53 R52 R51
[weights]
5.0 3.6 2.4 1.8 2.2 3.5 3.5 2.2 1.8 2.4 3.6 5.0
4.0 1.6 1.2 1.0 1.0 3.0 3.0 1.0 1.0 1.2 1.6 4.0
5.0 3.4 2.6 2.2 1.8 4.0 4.0 1.8 2.2 2.6 3.4 5.0
1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0
layouts
all the defined stock layouts in the config.txt file have added a null thumb row, e.g..
[layouts]
>>Qwerty
# q w e r t y i o u p #
é a s d f g h j k l ; '
è z x c v b n m , . / -
# # # # # # # # # # # #
with BEAKL as..
>>BEAKL Zi
# z y o u : g d n m x #
# q h e a . c t r s w #
# j - ' k , b p l f v #
# # # # i # # # # # # #
>>BEAKL Wi
# : y o u - g d n m z #
# q h e a w c t r s v #
# j , . k ' b p l f x #
# # # # i # # # # # # #
penalties
in the config.txt file have the following thumb-finger, finger-thumb combinations and top row thumb jumps added..
[penalties]
,same_row,row_jump1,row_jump2,row_jump3
ii, 2.5, 3.5, 4.5, 0.0 // same finger
im, 0.5, 1.0, 2.0, 0.0
ir, 0.5, 0.8, 1.5, 0.0
ip, 0.5, 0.8, 1.1, 0.0
mi, -1.5, -0.5, 1.5, 0.0 // inward roll
mm, 2.5, 3.5, 4.5, 0.0 // same finger
mr, 0.5, 1.0, 2.0, 0.0
mp, 0.5, 0.8, 1.5, 0.0
ri, -1.5, -0.5, 1.5, 0.0 // inward roll
rm, -2.0, -0.5, 1.2, 0.0 // inward roll
rr, 2.5, 3.5, 4.5, 0.0 // same finger
rp, 1.0, 1.5, 2.5, 0.0
pi, -1.0, 0.0, 1.0, 0.0 // inward roll
pm, -1.0, 0.0, 1.5, 0.0 // inward roll
pr, -1.0, 0.0, 1.5, 0.0 // inward roll
pp, 3.0, 4.0, 5.5, 0.0 // same finger
it, 0.5, 1.5, 0.5, 1.0
ti, 0.5, 1.5, 0.5, 1.0
mt, 0.5, 1.5, 0.5, 1.0
tm, 0.5, 1.5, 0.5, 1.0
rt, 0.5, 1.5, 0.5, 1.0
tr, 0.5, 1.5, 0.5, 1.0
pt, 0.5, 1.5, 0.5, 1.0
tp, 0.5, 1.5, 0.5, 1.0
tt, 2.0, 3.0, 2.0, 2.5 // same finger
Thumb key penalties have been set to relatively low and consistent values because the thumb itself, unlike the fingers, never shifts position while typing words. Thumb row_jump1 and row_jump3 penalties reflect finger travel from row_jump2—the home row at rest finger position.
default weighting
given the augmented default configuration above, the default weights and penalties yield the following comparison..
| en | fr | es | de | |
|---|---|---|---|---|
| BEAKL Zi | 64.17 | 63.18 | 66.14 | 66.73 |
| BEAKL Wi | 64.25 | 62.70 | 65.37 | 67.00 |
| MTGAP | 64.48 | 69.73 | 66.14 | 67.15 |
| BEAKL 19bis | 64.51 | 66.87 | 64.45 | 67.92 |
| Colemak DHm mod | 64.54 | 67.32 | 64.18 | 64.70 |
| Colemak DHm | 64.66 | 67.88 | 64.18 | 64.70 |
| MTGAP 2.0 | 64.91 | 67.21 | 64.01 | 64.98 |
| MTGAP “ergonomic” | 64.99 | 69.18 | 64.81 | 66.23 |
| White | 65.10 | 73.60 | 68.09 | 66.50 |
| Kaehi | 65.56 | 70.35 | 65.92 | 67.83 |
| Colemak DH | 65.71 | 69.32 | 65.67 | 65.43 |
| Workman | 65.83 | 71.42 | 66.85 | 66.93 |
| MTGAP “standard” | 65.84 | 68.35 | 64.43 | 66.78 |
| Soul mod | 65.89 | 68.96 | 64.71 | 64.38 |
| BEAKL 19 | 65.98 | 70.36 | 66.12 | 66.99 |
| MTGAP “shortcuts” | 66.02 | 68.24 | 62.72 | 65.44 |
| BEAKL 19 Opt French | 66.32 | 67.12 | 65.85 | 65.31 |
| Oneproduct | 66.44 | 73.48 | 68.07 | 68.45 |
| Hands down | 66.64 | 68.97 | 66.10 | 63.14 |
| MTGAP “Easy” | 66.78 | 68.63 | 64.55 | 64.97 |
| Colemak | 67.15 | 68.40 | 65.37 | 67.77 |
| Niro mod | 67.41 | 70.47 | 66.58 | 67.71 |
| BEAKL 15 | 67.43 | 71.86 | 66.64 | 69.37 |
| Three | 68.23 | 73.43 | 69.46 | 71.00 |
| Norman | 68.34 | 74.01 | 71.24 | 69.86 |
| ASSET | 68.88 | 69.42 | 66.69 | 70.35 |
| Notarize | 69.45 | 70.76 | 67.68 | 69.21 |
| qgmlwyfub | 70.83 | 75.87 | 70.24 | 72.16 |
| Carpalx | 71.02 | 76.25 | 70.86 | 74.00 |
| Qwpr | 71.69 | 73.36 | 69.44 | 73.07 |
| Coeur | 72.29 | 67.35 | 67.07 | 71.04 |
| Bépo keyberon | 72.79 | 68.15 | 68.53 | 72.07 |
| Minimak-8key | 72.94 | 74.81 | 71.94 | 75.20 |
| Bépo 40% | 73.20 | 67.97 | 68.54 | 73.10 |
| Dvorak | 73.74 | 78.15 | 75.34 | 74.59 |
| Neo | 76.31 | 76.37 | 74.56 | 71.62 |
| Qwertz | 98.56 | 98.12 | 93.66 | 98.31 |
| Qwerty | 100.00 | 98.90 | 92.45 | 99.74 |
| Azerty | 105.44 | 104.18 | 102.40 | 102.81 |
Not surprisingly, anything but QWERTY and its derivatives is a good choice for anyone wishing a better layout to type with. Adhering to these common finger weights and penalties, all the other keyboards perform comparably better—choose the one that feels best! Colemak, in this regard, is a good choice for many due to the less radical rearrangement of keys from QWERTY.
i was, quite honestly, not expecting the BEAKL variants to score so well given their differing design objectives. In particular, the ranking of the Thumb I variants perhaps reflects directly the benefit of the thumb key assignment.
beakl weighting
applying my personal BEAKL pinkie finger and extended index finger weights—these differ slightly from the original BEAKL weightings per my hand dexterity, maintaining the severe pinkie finger penalties whilst relaxing the extended index finger penalties a touch—to emphasize usage of the center ring-middle-index finger columns..
[weights]
16.0 12.0 2.0 1.0 1.0 4.0 4.0 1.0 1.0 2.0 12.0 16.0
9.0 5.0 1.0 0.5 0.5 1.5 1.5 0.5 0.5 1.0 5.0 9.0
10.0 7.0 2.5 2.0 1.5 5.0 5.0 1.5 2.0 2.5 7.0 10.0
0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5
with further adjusted (more severe) ring-pinkie, pinkie-ring penalties—again per my hand dexterity, disliking those particular finger rolls..
[penalties]
,same_row,row_jump1,row_jump2,row_jump3
ii, 2.5, 3.5, 4.5, 0.0 // same finger
im, 0.5, 1.0, 2.0, 0.0
ir, 0.5, 1.0, 1.5, 0.0
ip, 0.5, 1.0, 1.5, 0.0
mi, -1.5, -0.5, 1.5, 0.0 // inward roll
mm, 2.5, 3.5, 4.5, 0.0 // same finger
mr, 0.5, 1.0, 2.0, 0.0
mp, 1.0, 1.5, 2.5, 0.0
ri, -1.5, -0.5, 1.5, 0.0 // inward roll
rm, 1.0, 1.5, 2.5, 0.0 // inward roll
rr, 3.0, 4.0, 5.0, 0.0 // same finger
rp, 3.0, 4.5, 7.0, 0.0
pi, -1.0, 0.0, 1.0, 0.0 // inward roll
pm, 1.0, 2.0, 3.0, 0.0 // inward roll
pr, 4.0, 5.0, 6.5, 0.0 // inward roll
pp, 5.0, 6.0, 8.0, 0.0 // same finger
it, 0.5, 1.5, 0.5, 1.0
ti, 0.5, 1.5, 0.5, 1.0
mt, 0.5, 1.5, 0.5, 1.0
tm, 0.5, 1.5, 0.5, 1.0
rt, 0.5, 1.5, 0.5, 1.0
tr, 0.5, 1.5, 0.5, 1.0
pt, 0.5, 1.5, 0.5, 1.0
tp, 0.5, 1.5, 0.5, 1.0
tt, 2.0, 3.0, 2.0, 2.5 // same finger
yield the following comparison..
| en | fr | es | de | |
|---|---|---|---|---|
| BEAKL Wi | 45.43 | 44.73 | 45.54 | 50.66 |
| BEAKL Zi | 48.59 | 45.93 | 46.77 | 53.37 |
| BEAKL 15 | 55.14 | 69.54 | 53.16 | 58.94 |
| BEAKL 19bis | 60.84 | 58.45 | 53.67 | 63.48 |
| qgmlwyfub | 61.09 | 72.96 | 56.76 | 62.05 |
| Carpalx | 61.38 | 73.88 | 57.50 | 64.56 |
| BEAKL 19 | 61.58 | 74.00 | 54.91 | 56.25 |
| BEAKL 19 Opt French | 64.39 | 51.34 | 48.79 | 64.96 |
| Neo | 65.58 | 79.58 | 66.37 | 62.07 |
| Hands down | 69.44 | 86.65 | 80.16 | 66.41 |
| Kaehi | 69.61 | 85.44 | 67.53 | 75.30 |
| Colemak DHm mod | 69.98 | 80.47 | 80.01 | 61.49 |
| Colemak DHm | 70.12 | 77.87 | 80.01 | 61.49 |
| Norman | 70.82 | 78.69 | 77.77 | 67.92 |
| Colemak | 70.88 | 77.76 | 80.27 | 63.84 |
| MTGAP “ergonomic” | 71.36 | 76.03 | 68.01 | 66.06 |
| MTGAP “shortcuts” | 71.62 | 78.58 | 65.70 | 73.71 |
| Soul mod | 71.67 | 80.98 | 81.66 | 58.75 |
| Minimak-8key | 71.96 | 86.21 | 84.73 | 67.09 |
| Workman | 72.29 | 86.97 | 84.71 | 71.27 |
| Colemak DH | 72.77 | 81.57 | 83.78 | 63.38 |
| ASSET | 73.21 | 82.93 | 81.14 | 74.04 |
| Notarize | 74.06 | 86.02 | 84.44 | 71.43 |
| MTGAP “standard” | 74.42 | 78.15 | 71.12 | 67.46 |
| MTGAP 2.0 | 75.12 | 80.38 | 70.27 | 61.15 |
| Niro mod | 75.49 | 84.41 | 87.40 | 65.78 |
| Oneproduct | 76.22 | 87.29 | 72.67 | 79.35 |
| Three | 76.35 | 86.43 | 78.18 | 71.19 |
| MTGAP “Easy” | 78.32 | 91.93 | 89.26 | 69.75 |
| MTGAP | 78.38 | 81.47 | 70.37 | 78.75 |
| White | 80.66 | 92.86 | 86.25 | 74.97 |
| Qwpr | 82.50 | 97.79 | 94.37 | 87.19 |
| Dvorak | 88.14 | 106.29 | 101.62 | 85.94 |
| Coeur | 88.78 | 83.13 | 81.30 | 95.43 |
| Bépo keyberon | 91.88 | 81.64 | 88.00 | 103.14 |
| Bépo 40% | 92.41 | 81.46 | 88.00 | 104.14 |
| Qwerty | 100.00 | 112.05 | 105.83 | 94.46 |
| Qwertz | 102.27 | 112.16 | 109.02 | 92.28 |
| Azerty | 127.97 | 135.17 | 141.94 | 114.45 |
Unsurprisingly, weights and penalties reflecting BEAKL objectives favour these layouts. Interestingly, the ranking of the Carpalx variants also improved with these values, illustrating its claimed reduced pinky finger usage. With personally adjusted values then, kb-layout-evaluation can be used to filter suitable stock keyboard layout candidates for consideration—versus the notion of a “best” layout.
BEAKL Wi fares well with its BEAKL Zi heritage, illustrating the potential advantage thumb clusters have when utilized for optimizing the base layer. Having said that, some hold the view that the thumb should not be assigned an alpha key, being slightly less nimble than fingers (while indisputably stronger and, perhaps, because of that).
Keyboard layouts are all about trade-offs and, as stated earlier, it all ultimately comes down to what feels best for one’s particular set of hands. i happen to abhor pinkie finger usage and have no problem with the Thumb I, others will feel differently and have differing fingering priorities.
Lots of layout choices out there for sure..