Dr. Edward Morbius ⭕ is a user on mastodon.cloud. You can follow them or interact with them if you have an account anywhere in the fediverse. If you don't, you can sign up here.


"Design" as practiced today by computer people tends to be heavily based on the idea of negative space: that good design is what's NOT in a system, and by extension, what is NOT ALLOWED TO BE ADDED to the system by a user.

A "design-heavy" system, then, is inevitably highly restrictive about user actions, lest they "ruin the cool design" by adding their own desired features that "make it messy".

But users WANT control over their own space.

Is there a "design" that empowers users?

@pnathan Sadly the CLI concept didn't extend even as far as the X Windowing System.

There may be a reason for that.

@natecull @pnathan What of stuff like Applescript?

(NB: I'm aware it exists. I've not used it myself and don't know its capabilities. A point which might factor into any possible response.)

@dredmorbius @natecull I would say the ergonomics of applescript are such that it's not full of use.

I didn't dink much with it because it didn't relate to the GUI paradigm that I worked in too well and I could already write bash etc by the time I encountered it.

@pnathan @natecull @dredmorbius hypercard was one of my early inroads to building software, so i used hypertalk some - similar vibes, i think?

hypercard was an amazing thing and i still miss aspects of it, but language-wise hypertalk suffered a lot from mistaken ideas about english-like ergonomics, i think.

@dredmorbius @natecull @pnathan
re: X, many (most?) of the linux desktop users i know use it to multiplex a bunch of terminals, and people tend to script things within the environment, so i'm not sure it's accurate that the cli didn't carry over.

@brennen @pnathan @dredmorbius

right, but that's not using X *as X* or to interact *with* CLI. It's using X just as a set of terminals for CLI that's completely unaware of X.

X, the system (as I understand it), does not use CLI. It sends and receives messages of its own devising and format but doesn't expose these to the user or use the CLI or pipe system to do it (except for some command options for programs which are mostly now ignored)

Tcl/tk, though, I think does use Unix pipes?

@natecull @dredmorbius @pnathan i mean, X is a windowing environment, so i dunno how using it to contain a bunch of windows isn't using it as that.

and yeah, X is a giant weird beast and i've never actually written code against it in any meaningful way, but i use stuff like xclip, wmctrl, dmenu, rofi, etc., to control facets of it pretty routinely, so it's at least situated in a cli & scripting-driven environment that i use to control my own space.

@brennen @pnathan @dredmorbius

Well, because X is about *graphical components* of which 'windows' are just one tiny, tiny subset

Like you might want your program to accept input from and send data to buttons? Grids? Charts? Images? Textboxes?


None of that is even really possible with the 'a simulated VT-100 running just inside a window' use case of X.

So CLI is not interfacing with X 'as X', just X 'as a VT-100'.

@dredmorbius @pnathan @brennen all of that though is probably possible with Tcl/tk and maybe with some of the oldschool CLI tools you mention? (which really, really don't get much press at all even among hardcore Linux fans)

like to properly represent the state of X as CLI I think you'd need at least an object model? cos things on the screen ARE objects, they have persistence and state

Maybe all I want is a modern Tcl/tk that works with all the shiny GUI toolkits and OOP systems?

@natecull @pnathan @brennen NB: I'm massively hampered in all this discussion principally by /never having really grokked GUI application design in the first place/. My sense of "application" has always been "something that works on a stream of inputs or polls regularly for state", but not "and presents this with a bow and cherry on top".

I could use pointers on some good books on GUI UI/UX. Don Norman seems to be one source. Brett Victor another.

@dredmorbius @natecull @pnathan modulo a handful of toy projects and tiny utilities, every "GUI" i've ever built has lived inside a web browser, so i s'pose i'm in much the same boat.

Dr. Edward Morbius ⭕ @dredmorbius

@brennen @natecull @pnathan I ... have markedly better CSS chops than I care to discuss in polite, or impolite, society. No JS though. Yet.

@pnathan @natecull @brennen Virtually all of that picked up out of necessity trying to salvage my sanity and/or vision from the present state of HTML maldesign.

@dredmorbius @pnathan @brennen

It seems quite hard to think of how to encapsulate the IO flow and state of a GUI component.

ie the input stream could be a stream of events (mouse/keyboard or underlying widgets, data model) and a similar output stream... but it all seems a bit confused and intertwined compared to a simple script that reads a stream of records and writes a stream of updated records.

Could *maybe* separate the IO flow into 'channels' like 'standard output' and 'standard error'

@brennen @pnathan @dredmorbius

and maybe MVC could be pictured at toplevel as:

(stream of view-modification requests + stream of model updates) --> UI component --> (stream of view-updates + stream of model updates)

but it still seems not right. It's a whole nonlinear *data* flow. And that nonlinearity is what makes it hard to modularise. Too many wires snaking out in all directions, all needing precisely matching sockets.

@natecull @dredmorbius @pnathan @brennen don't try to map unix pipe concepts on a UI, the only place that works is Android intents et al

I only skimmed replies, but AppleScript was going in the right direction. I wonder what smalltalk did for representing object passing?

@calvin @natecull @pnathan @brennen If you look at my "Tyranny of the Minimum Viable User" piece, you'll see I discuss that specifically.


There is also the distinction between "nodal" utilities and "complexity hubs". The latter do stdin/stdout poorly, if at all (though some do in fact do it). Most complexity hubs have *multiple* inputs and outputs.

@natecull @pnathan @brennen Interesting case in point: I've always found awk scripts far more intuitive than Excel spreadsheets. Even for simple stuff.

For sufficiently simple stuff, I write utilities that I'd otherwise use a spreadsheet for. E.g., summing a sequence, or generating univariate moments.

@dredmorbius @natecull @pnathan @brennen see, people are REALLY underestimating excel here

excel is a funcprog environment that

1., normal people use, and normal people use to great effect to get things done

2. actually offers a non-linear way to do programming; instead of a linear program, you operate on cells of a 2D space containing either data or formulae, and operating on ranges is easy

adding VBA was a terrible mistake that added complexity and inelegance

@calvin @dredmorbius @pnathan @brennen right, Excel and its ancestors (Visicalc) are a great example (I think) of what's now being called 'functional reactive programming'.

What if the whole GUI was like an Excel spreadsheet, and widgets were cells? They have a value, the value can be changed, it doesn't care how the value changes but if it does, that change propagates to all observers.

@natecull @dredmorbius @pnathan @brennen Mathematica has elements of this, but it's aiming more towards REPL-document hybrid

@brennen @pnathan @dredmorbius @calvin

the trick is how to remove irrelevant detail ('the user moved the mouse') as soon as possible, and pass on the relevant detail ('African_swallow.airspeed is now 42').

@calvin @natecull @pnathan @brennen I'm not contesting that Excel /isn't/ a functional programming environment.

I'm contesting that it's a /good/ FPE.

It is *VASTLY* too error-prone and difficult to debug. Not to mention awkward. OTOH, *it is often the only tool available since "real" programming tools are denied to front-office workers.*

(Including awk.)

Ray Panko, Univ. Hawaii, has studied Excel errors since the 1990s:

@dredmorbius @natecull @pnathan @brennen I don't think it means the idea is unsound though; on the contrary. There's a lot of ideas to take the spreadsheet and make it a far more useful tool than awk's programming equivalent of a face for radio

@calvin @natecull @pnathan @brennen Awk offers a line-of-code development model.

Excel offers a *grid of code* model. That's hard to see, visualise, debug, QC, validate, etc.

It's *possible* to write Very Robust Excel. It's not *easy* to do so.

I'll also note that I'm talking of my own personal use: I'm not saying that *everyone* who has an Excel-like task should use Excel. I'm saying that when I've got a simple "accumulate a store of values, compute something, kick out report", I prefer awk.

@dredmorbius @calvin @pnathan @brennen

I suspect this is because spreadsheets don't offer *grouping* constructs, which are pretty important for humans to organise information.

A 2D grid gets you a long way, but not really far enough.

If you had a sort of web of nodes which you could expand or shrink, and had a natural mapping TO a 2D grid if you wanted it...

@natecull @pnathan @calvin @dredmorbius a spreadsheet is a shittier-than-average database with what is a better-than-average representation layer for a whole bunch of people to be able to grasp it.

the achilles heel is how bound the physical structure of the data is to the representation & interface, but maybe that's what makes it approachable. there has to be some way for a serious db to be as approachable as excel is, but i don't know what it'd be.

@brennen @natecull @pnathan @calvin And yes, the notion that Excel integrates data storage, data manipulation, and data output, is pretty key.

MS Access took that a slight level higher, though it also had a remarkable propensity to suddenly detonate and destroy entire city blocks, without warning.

@natecull @dredmorbius @pnathan @brennen well, there are "sheets" but it's a bit clumsy

I think perhaps copying ideas from SQL for structure and drilldown would be interesting (another funcprog environment with good ideas, but clumsy tooling; SQL feels like funcprog if it were implemented by programmers who only used COBOL, and half the functions are missing - i'm convinced SQL is also why people swear of RDBMSes to the point of using Mongo et al)

@calvin @natecull @dredmorbius @pnathan put me in the camp of "syntax irregularity and the friction of defining new structures aside, sql is basically pretty good".

those caveats do matter, though. my hunch is that mostly people get lured into garbageware like mongo because sql rdbms systems make manipulating schema a difficult, largely out-of-band sort of task relative to most of their interface.

@brennen @natecull @dredmorbius @pnathan see, I always think talking about "schemaless" is a red herring when it comes to NoSQL - it's really about SQL as a query language & about making the DB just plain ol' serialization of Plain Ol' Objects - this is easy for a programmer, and it's a *schema*, typed language or not

SQL is only lovable if you're a 70s DBA. I'd want a modern RDBMS protocol of sorts, that makes queries and serialization easy - with strict schemas, since the other end has those

@brennen @pnathan @dredmorbius @calvin yeah, a database where defining a schema is an out-of-band operation seems a little like a functional programming language where you can't pass functions to functions.

One of the defining characteristics of data is that it exhibits recursive structure: things inside other things.

RDMBSes in the SQL model, however, are built assuming that there are certain Things (databases, tables) which May Not Ever Be Put Inside Other Things.

@natecull @brennen @pnathan @dredmorbius well, hierarchial DBMSes exist (a lot of NoSQL can be considered as such) but then you lose the advantages of RDBMSes. the problem is that mappings in SQL between tables are clumsy. what if RDBMSes had typed pointers between objects that made it easier to related between objects between tables?

@natecull @brennen @pnathan @calvin Having learnt RDBMS and various mainframe hierarchical data formats more-or-less simultaneously, I've been aware that There Is More Than One Way to Fuck Things Up.

Relational 2D data is convenient. It does /not/ fit all sets of circumstances. Normalising hierarchical data may or may not be possible. And often you're dealing with a cross of "what is present state" and "what is most recently-added transaction" (or "what is the full transaction history"?)

@calvin @pnathan @brennen @natecull There's also the realisation that many mainframe hierarchical formats appear to have been lifted straight off the underlying *paper* forms in use at the time.

At which point you realise that it was easier for the programmers *then* to simply duplicate the paper record in electronic form than to rationalise it.

@calvin @dredmorbius @pnathan @brennen and a sea of objects, or even dumb objectlike key-value data like JSON, doesn't have this restriction!

so it's naturally a lot simpler to work with 'an object full of objects full of objects'


'a database, which I gotta get my RDMBS administrator to spec a server and provision for me and define a schema which if I ever then change I gotta file a helpdesk request to do it and butwhooops the user just put a new data type in and crap...'

@natecull @calvin @pnathan @brennen Is JSON really a persistent data /store/?

Or is it a way of shuttling data between systems?

@dredmorbius @natecull @calvin @pnathan anything is a persistent data store if exists in the vicinity of enough half-educated programmers.

ANYWAY, this thread is a lot of fun but now i'm going to go fuck around with a pile of music gear i don't understand, which is probably also a good place to think dark thoughts about the state of interfaces and modularity.

@brennen @dredmorbius @natecull @pnathan for parting thoughts on spreadsheets, simmer on stuff like Quantrix, "3D" structured spreadsheets

@dredmorbius @calvin @pnathan @brennen

yes and yes.

I keep thinking that JSON is the new '80-column IBM card'.

It's terrible, but it's at least a defined 'plug shape and pinout'

so data stores are gonna assume it as the 'shape' of the data they store, just like 80-column screens and text files stayed the standard well into the disk and VDU era

@natecull @dredmorbius @pnathan @brennen I mean, JSON column types exist in everything from MySQL to DB2/400 (which is by far the weirdest SQL server I can think of)

@natecull @calvin @dredmorbius @pnathan @brennen To me, many of the problems of SQL arise because of how we use it: 1. Very restrictive with permissions. How much easier to solve problems if you can creat a ton of temporary tables. How much easier the queries. 2. Embedding it in apps. It’s like embedding vi in an app. Useless! But using it directly is cool. Like having awk and creating new files is cool. But better.

@kensanata i have probably linked this bit of rambling here before, but i've harbored related thoughts for a while: p1k3.com/2015/2/1/

@brennen The only point I’d like to add to that is that single user single file SQLite would answer many of the concerns, I think.

@kensanata yeah, perhaps. i do use (and like) sqlite for some stuff, but the limitations are real. i really haven't followed up on these thoughts by incorporating a relational db into much of my personal stuff, so i'm not honestly sure how much i'd feel those constraints if i just used sqlite for more things.

@kensanata @natecull @calvin @dredmorbius @pnathan @brennen I don’t understand your point 2. Why don’t you want programs to use SQL?

@wrenpile @kensanata (Retrying this message)

I think the argument is that SQL is a tool for working with and analyzing data like vi(m) is a tool for working with text files. So embedding SQL into a program artificially solves the problem by shoehorning in a tool's "interface" (magic strings) and at the same time introduces tons of second order concerns (sanitization, etc) as well as domain-specific middleware.

@cj @wrenpile @kensanata But applications have been built around RDBMS (and other) data stores ... since pretty much forever. If SQL *isn't* a general data language, than what is?

@dredmorbius @wrenpile @kensanata I think your statements/questions are orthogonal. SQL and RDBMS aren't coupled. No one is arguing against "SQL is a general data language". But is embedding SQL into binaries a good thing? Does that answer depend on ORM support for the language? Or sanitization libraries? Or sharding libs? I'd argue SQL-RDBMS coupling and SQL-as-a-general-data-language are orthogonal to these concerns.

@wrenpile I don’t mind embedding SQL in apps given that I don’t want two different ways to access data in a database. Being able to copy and paste SQL statements from source code and trying them in a general REPL is great for debugging. For people who come from a strict programming background (Java only for example) the embedded SQL (strings, no type safety, no compilation) is abhorrent. That’s why we have Java based QL now and it makes debugging hard. 😒

@kensanata Thanks for clarifying.

When you have a complex query with multiple joins and subqueries, having a REPL is vital not just for debugging, but also for prototyping.

As for Javaworld snobbery toward SQL, hey, SQL SELECTs are functional programming, and Java is … ?

@wrenpile Haha. Well, what can I say. At the office they are all gung-ho about getting rid of SQL statements and replacing them with a QL that generates the statements and thus guarantees type safety (database tables generated from interfaces and bind variables generated from the same interfaces). I tell them that this catches the kinds of errors I don't make and makes my life more difficult regarding the errors I do make, but it's not helping.

@kensanata Let me guess: in place of one big sql query yielding the answer you want, you end up with a bunch of small queries stitched together with many many imperative statements.

@wrenpile I haven't seen much of that, to be honest. We just translate big SQL statements into big nested Java function calls.

@natecull @calvin @dredmorbius @pnathan meh. i'm not really pining for more hierarchy in my datastores, though it has its place (hierarchical filesystems are certainly useful). my gripe is more that there's a bunch of friction to making new tables.

@brennen @calvin @natecull @dredmorbius @pnathan postgresql does a surprisingly good job at not being terrible, for a rdbms

@natecull @calvin @pnathan @brennen There are macros, worksheets, regions, etc. Though the groupings are predominantly on that 2D grid.

So it's not /all/ that bad.

And there are any number of CLI / scripting languages that promote Very Bad Habits. PHP is notorious for this. Perl can definitely get you there. I've even heard C occasionally causes problems.