The New Beginnings
This commit is contained in:
389
posts/2020-08-20-the-onyx-programming-language.md
Normal file
389
posts/2020-08-20-the-onyx-programming-language.md
Normal file
@@ -0,0 +1,389 @@
|
||||
---
|
||||
title: The Onyx Programming Language
|
||||
location: Moscow, Russia
|
||||
---
|
||||
|
||||
In the [previous article](/posts/2019-08-01-system-programming-in-2k20) I successfully justified my desire to build yet another system programming language.
|
||||
Unlike _others_, I want to do it right from the very beginning.
|
||||
Meet Onyx!
|
||||
|
||||
<!-- excerpt -->
|
||||
|
||||
<h2>Table of Contents</h2>
|
||||
|
||||
${toc}
|
||||
|
||||
## The Onyx programming language
|
||||
|
||||
|
||||
|
||||
Onyx is a general-purpose statically typed programming language suitable both for application and system programming.
|
||||
|
||||
### Top-Down Features
|
||||
|
||||
Onyx syntax is inspired by **C-family** languages, such as Ruby, C++ and even <s>Rust</s> <small>I'm sorry</small>.
|
||||
|
||||
Onyx imposes powerful **inference** mechanisms.
|
||||
The rule is generally "infer unless ambiguous", with meaningful defaults.
|
||||
|
||||
Onyx is a **memory-safe** language.
|
||||
There are multiple levels of enforced safety, with `unsafe` being the minimum safety level allowing for extreme optimizations.
|
||||
This opens great opportuinites for powerful abstractions.
|
||||
|
||||
Unlike [_other_](https://www.rust-lang.org/){.secret-link} languages, Onyx treats **pointer arithmetic** as a first-class use-case, but with memory safety.
|
||||
The concept is expressed in raw but typed pointers containing a memory scope.
|
||||
For example, it is not possible to safely pass a local pointer to an outer scope.
|
||||
Moreover, pointers preserve low-level features like address spacing and alignment.
|
||||
|
||||
It is extremely easy to do **interoperability** in Onyx.
|
||||
In fact, Onyx is similar to C++ in this sense: C code is considered a part of the program (with minor differences), and it is simple to export Onyx code as a shared library.
|
||||
|
||||
Programs written in Onyx are **cross-platform** by default in the sense of that there are no target-dependent features in the language itself: no threading, no memory control.
|
||||
But fear not, thanks to powerful abstractions, these are likely to be already implemented by someone else!
|
||||
|
||||
Onyx introduces **powerful macros** written in Lua.
|
||||
It allows to re-use existing Lua code and have full access to the compilation context thanks to easy debugging with Lua.
|
||||
|
||||
Onyx has **<u>[simple-to-understand](https://twitter.com/vladfaust/status/1299116755596566528){.secret-link}</u> lifetime and moving** concepts naturally built into the language.
|
||||
Instead of fighting with a borrow checker, simply get an address of a variable: a compiler would not allow you to mis-use it.
|
||||
|
||||
Classes may have a finalizer defined and thus have **automatic resource control**.
|
||||
|
||||
Onyx implements real **traits** as [composable units of behaviour](https://en.wikipedia.org/wiki/Trait_(computer_programming)) thanks to powerful function management tools like aliasing, implementation transferring, un-declaring and renaming.
|
||||
|
||||
Classes and traits together impose **object-oriented** capatibilites of the language.
|
||||
|
||||
Onyx has a concept of **generics**.
|
||||
Specializations of generic types may have different members, and evaluate delayed macros.
|
||||
Specializations of functions with generic arguments may return differnt values and also evaluate delayed macros.
|
||||
|
||||
Functions may be **overloaded** by arguments and return values.
|
||||
|
||||
Onyx has a concept of **annotations**, which may be applied to variables and functions.
|
||||
|
||||
The language defines a set of now-commonly used **arithmetic types**, including SIMD vectors, matrices and tensors, floating and fixed binary and decimal numbers, brain and tensor floats, ranges and ratios.
|
||||
|
||||
Onyx contains a number of **utility types**, such as unions, variants, tuples, anonymous structs, lambdas and runnable blocks of code.
|
||||
|
||||
Exceptions are designed to be truly **zero-cost** to enable exception flow in Onyx programs.
|
||||
|
||||
### Examples
|
||||
|
||||
Let's jump into some code samples.
|
||||
This would give a brief overview of how Onyx programs look like.
|
||||
|
||||
#### Hello, world!
|
||||
|
||||
This is a very basic program written in Onyx:
|
||||
|
||||
```text
|
||||
import "stdio.h"
|
||||
|
||||
export int main() {
|
||||
final msg = "Hello, world!\0"
|
||||
unsafe! $puts(&msg as $char*)
|
||||
return 0
|
||||
}
|
||||
```
|
||||
|
||||
In the example, a C header named `"stdio.h"` was imported into the Onyx program.
|
||||
Now Onyx is aware of assembly functions declared by this header.
|
||||
|
||||
Later on, one of these functions, `puts` is called directly from Onyx.
|
||||
An Onyx compiler can not give any safety guarantees in regard to called assembly functions, therefore the call must be wrapped in an `unsafe!` statement.
|
||||
|
||||
If something weird happens, a developer may simply `grep` the program source code for `unsafe!` statement to quickly narrow to potentially hazardous areas of code.
|
||||
|
||||
Imported entities are referenced with preceding `$` symbol to distinguish them from those declared in Onyx context.
|
||||
|
||||
A constant named `msg` is defined by the `final` statement.
|
||||
The type of `msg` is inferred to be `String<UTF8, 14>`, i.e. a [UTF-8](https://en.wikipedia.org/wiki/UTF-8)-encoded array of [code units](https://en.wikipedia.org/wiki/Code_unit) containing 14 elements.
|
||||
|
||||
Then, address of the `msg` constant is taken.
|
||||
The resulting object of taking an address would be `String<UTF8, 14>*lr0`, which is a shortcut to `Pointer<Type: String<UTF8, 14>, Scope: :local, Readable: true, Writeable: false, Space: 0>`.
|
||||
|
||||
Do not be intimidated, though!
|
||||
Thanks to inference, shortucts and meaningful defaults, you'll rarely have to use full types.
|
||||
|
||||
The pointer to `msg` is then coerced to C type `char*`.
|
||||
Such a coercion would be unsafe, and a compiler would normally panic.
|
||||
|
||||
However, the coercion is already within unsafe context itself thanks to the wrapping `unsafe!` statement.
|
||||
No need to write `unsafe!` again.
|
||||
|
||||
The program above is normally compiled by an Onyx compiler, such as [`fnxc`](https://github.com/fancysofthq/fnxc), into an object file.
|
||||
The object file declares the exported `int main(void)` prototype, which must be pointed to at as a entry function by a system linker.
|
||||
Thankfully, this tedious operation is likely to be automatically handled by a higher-level build tool, such as [`fnx`](https://github.com/fancysofthq/fnx).
|
||||
|
||||
Note that Onyx does not have any implicit `__onyx_main` function, which effectively restricts non-trivial automatic static variable initialization and finalization.
|
||||
But in return it makes the emitted code predictable and portable.
|
||||
|
||||
#### Using a standard library
|
||||
|
||||
An Onyx compiler is not required to implement any sort of OS-specific standard library.
|
||||
Instead, the standard library Standard is specified elsewhere (spoiler alert: by [the Onyx Software Foundation](#the-onyx-software-foundation)).
|
||||
|
||||
A standard library is ought to be used as a common package and required as any other from your code.
|
||||
Again, by default an Onyx program does not depend on any OS-specific features.
|
||||
|
||||
The example above could be abstracted into this when using a standard library implementation:
|
||||
|
||||
```text
|
||||
require "std"
|
||||
|
||||
export void main() {
|
||||
let msg = "Hello, world!\0"
|
||||
|
||||
try
|
||||
Std.puts(&msg)
|
||||
catch
|
||||
Std.exit(1)
|
||||
end
|
||||
}
|
||||
```
|
||||
|
||||
Now, the code is perfectly safe.
|
||||
Even passing of `&msg` is allowed, because `Std.puts` has an overload accepting a `String*cr`, i.e. a read-only pointer with _caller_ scope, and a pointer with _local_ scope may be safely cast to to _caller_ scope upon passing to a function!
|
||||
|
||||
Also note that `msg` is now a variable, as it is defined with `let` statement.
|
||||
Taking address of `msg` would return `String<UTF8, 14>*lrw0`.
|
||||
Notice the `w` part?
|
||||
The pointer is now writeable.
|
||||
And it is perfectly legit to pass a writeable pointer as a read-only argument: it would be coerced down to a read-only pointer within the callee.
|
||||
|
||||
#### Exceptions
|
||||
|
||||
We had to wrap the `Std.puts` call into the `try` block, as it could throw some system exception.
|
||||
The cause is that an `export`ed function must guarantee to never throw an exception, that's why we wrapped it.
|
||||
The `Std.exit` function is declared as `nothrow`, so we can leave it as-is.
|
||||
|
||||
But what if we wanted to inspect the backtrace of the possible exception?
|
||||
Well, the language Standard states that a backtrace object must implement `Endful<Location>` trait.
|
||||
This is a truncated source code of the trait:
|
||||
|
||||
```text
|
||||
struct Location
|
||||
val path : String*sr # A static pointer
|
||||
val row, col : UBin32
|
||||
end
|
||||
|
||||
trait Endful<T>
|
||||
decl push(value: T)
|
||||
decl pop() : T
|
||||
decl pop?() : T?
|
||||
end
|
||||
```
|
||||
|
||||
Let's implement some `Stack` type to hold the backtrace.
|
||||
|
||||
::: spoiler ⚠️ A big chunk of code!
|
||||
|
||||
```text
|
||||
# A stack growing upwards in memory.
|
||||
#
|
||||
# The `~ %n` part means "accept a natural
|
||||
# number literal as a generic argument".
|
||||
class Stack<Type: T, Size: Z ~ %n>
|
||||
# This class derives from this trait.
|
||||
derive Endful<T>;
|
||||
|
||||
# Define two empty structs.
|
||||
struct Overflow;
|
||||
struct Underflow;
|
||||
|
||||
# Do not finalize this variable
|
||||
# in the end of stack lifetime.
|
||||
#
|
||||
# `@[NoFinalize]` is application
|
||||
# of an unsafe annotation.
|
||||
unsafe! @[NoFinalize]
|
||||
final array = unsafe! uninitialized T[Z]
|
||||
|
||||
# A getter makes the variable read-only
|
||||
# from outside, but writeable inside.
|
||||
get size : Size = 0
|
||||
|
||||
# A class, unlike a struct, does
|
||||
# not have a default initializer.
|
||||
def initialize();
|
||||
|
||||
# But class allows to
|
||||
# define a finalizer!
|
||||
def finalize()
|
||||
# Only those stack elements which
|
||||
# are alive shall be finalized.
|
||||
size.times() -> unsafe! @finalize(array[&])
|
||||
end
|
||||
|
||||
# Push a value into the stack.
|
||||
# Throws in case of stack overflow.
|
||||
# It implements `Endful<T>:push`.
|
||||
impl push(val)
|
||||
# The scary operator is expanded to
|
||||
# `size <<= size ^+ 1`, meaning
|
||||
# "push-assign to `size` a saturated
|
||||
# sum of it with 1".
|
||||
#
|
||||
# Push-assignment returns the old
|
||||
# value instead of the new one.
|
||||
if (size ^+<<= 1) < Z
|
||||
# `<<-` moves the value from `val`
|
||||
# into the array, "ejecting" the
|
||||
# old array value. But at this point,
|
||||
# the old array value is already
|
||||
# finalized, so we explicitly disable
|
||||
# it finalization here.
|
||||
unsafe!
|
||||
@nofinalize(array[size - 1] <<- val)
|
||||
end
|
||||
else
|
||||
throw Overflow()
|
||||
end
|
||||
end
|
||||
|
||||
# Pop a value from the stack.
|
||||
# Throws in case of stack underflow.
|
||||
#
|
||||
# Note the alternative syntax to
|
||||
# reference the declaration.
|
||||
impl ~Endful<T>:pop()
|
||||
return pop?() ||
|
||||
throw Underflow()
|
||||
end
|
||||
|
||||
# Pop a value from the stack
|
||||
# if it is not empty.
|
||||
# Returns `Void` otherwise.
|
||||
impl nothrow pop?()
|
||||
# Expands to `size <<= size ^- 1`.
|
||||
if size ^-<<= 1 > 0
|
||||
# We don't copy the array element,
|
||||
# but move it from the array.
|
||||
#
|
||||
# A copy of bytes of the element are
|
||||
# preserved in the array, but we
|
||||
# consider it already dead, a corpse.
|
||||
#
|
||||
# That's why we don't finalize it
|
||||
# in the `push` implementation.
|
||||
return unsafe! <-array[size]
|
||||
else
|
||||
return Void
|
||||
end
|
||||
end
|
||||
end
|
||||
```
|
||||
|
||||
Oof, sorry for such a seemingly complex piece of code.
|
||||
But I had to do it, sooner or later!
|
||||
|
||||
:::
|
||||
|
||||
But wait!
|
||||
Thankfully, the language already comes with a `Stack` implementation, so we don't need to write it in our code.
|
||||
|
||||
A `try` statement has optional `with` clause accepting a pointer to a `Endful<Location>` implementation.
|
||||
The example above may be rewritten like this:
|
||||
|
||||
```text
|
||||
require "std"
|
||||
|
||||
export void main() {
|
||||
let msg = "Hello, world!\0"
|
||||
final backtrace = Stack<Location, 32>()
|
||||
|
||||
try with &backtrace
|
||||
Std.puts(&msg)
|
||||
catch
|
||||
while final loc = backtrace.pop?()
|
||||
# Could've also made use of
|
||||
# `loc.row` and `loc.col`...
|
||||
Std.puts(loc.path)
|
||||
end
|
||||
catch
|
||||
# An unrecoverable error ☹️
|
||||
Std.exit(1)
|
||||
end
|
||||
}
|
||||
```
|
||||
|
||||
Now we can inspect the exception backtrace!
|
||||
|
||||
#### An HTTP server example
|
||||
|
||||
It is considred a good tone to demonstrate on how to build a simple echoing HTTP web server in your language.
|
||||
This would also be my "sorry" for the big-ass `Stack` implementation above.
|
||||
|
||||
The thing is that running a web server is architecturally different on different target platforms.
|
||||
An implementation on Linux could make use of raw sockets, an implementation on Windows could make use of the win32 `"http.h"` header etc.
|
||||
|
||||
Therefore, the standard library would not contain a web server implementation.
|
||||
Instead, some third-party package should be used, which would inevitably be a plenty of!
|
||||
|
||||
Let's imagine we've found one satisfying our needs.
|
||||
That's how it could look like:
|
||||
|
||||
```text
|
||||
require "std"
|
||||
require "http" from "mypkg"
|
||||
|
||||
export int main () {
|
||||
final backtrace = Stack<Location, 32>()
|
||||
|
||||
try with &backtrace
|
||||
final server = HTTP::Server()
|
||||
|
||||
server.get("/") ~> do |env|
|
||||
# Read the request into a
|
||||
# local `Std::Twine` instance.
|
||||
final body = env.request.read()
|
||||
|
||||
# Write the twine
|
||||
# into the response.
|
||||
env.response << body
|
||||
end
|
||||
|
||||
server.listen("localost", 3000)
|
||||
catch |e|
|
||||
Std.puts("Caught \@{{ e }}\n")
|
||||
|
||||
while final loc = backtrace.pop?()
|
||||
Std.cout << "At " <<
|
||||
loc.path << ":" <<
|
||||
loc.row << ":" << "\n"
|
||||
end
|
||||
|
||||
Std.exit(1)
|
||||
catch
|
||||
# Unrecoverable error 👿
|
||||
Std.exit(2)
|
||||
end
|
||||
}
|
||||
```
|
||||
|
||||
Thanks to powerful abstractions and type inference, you won't need to manualy use `socket` each time you want to spin up a web server on Linux!
|
||||
|
||||
Oh, by the way, did you notice the `\@{{ e }}`\* thing?
|
||||
It was me, <s>Dio</s> macro!
|
||||
|
||||
\* <small>TODO: Fix this HTML weirdness.</small>
|
||||
|
||||
#### Macros
|
||||
|
||||
TODO: Example on macros.
|
||||
Simple macros.
|
||||
Breakpoints in compile-time, code generation based on external configurations, e.g. of ORM models from SQL migration files.
|
||||
|
||||
## The Onyx Software Foundation
|
||||
|
||||
TODO: List all the standards.
|
||||
Official standard development process with RFSs, voting, community champions.
|
||||
Canonical package hosting platform with built-in funding based on the source-on-demand model.
|
||||
|
||||
<!-- TODO: Link the system programming article for justification.
|
||||
|
||||
TODO: Describe Onyx features, why it's better than other languages.
|
||||
|
||||
TODO: Describe NXSF.
|
||||
|
||||
TODO: Link the donating page. -->
|
||||
Reference in New Issue
Block a user