The Stone DB is consuming a lot of my time but its moving forward pretty nice... current single thread (in-RAM) imports run at 10 million nodes per second using an average node payload of 40 bytes (which is longer than the average I'd typically use). The majority of the time is spent verifying internal dataset integrity and memory copying.
it takes 3 seconds to basically grab all available process RAM (2GB ) and create 30 million data nodes. 1 million nodes takes 50ms on average, I'm getting pretty flat scaling so far, which is a very good sign . note that the data is completely memory copied into the DB . I'm not pointing to the original import data.
all of these benchmarks, are not even using a dedicated import function... this is like the worst case scenario for import. its a dumb FOR loop using a fully bounds checking single insert-node() function.... if I did an import loop which only does the bounds checking and stores counters in a loop, I likely can scale the import a lot.
I'm now starting work on the higher level interfaces, basically creating database setup on the fly and hopefully by friday I should have the file i/o started.
maybe next week I'll start to see how I can create a native Stone DB interface for R3.
nice tech you are doing there maxim. count me in for some big data tests. i never used graph DBs before but would like to give it a try for a non scalable setup, suboptimal solved via simple key traversal stored into a nosql core currently.
- Floating point numbers are now parsed and loaded as file! types, so external data with floats can at least be loaded and the numbers can be detected, so they could be processed further by your own functions.
red>> load-JSON "6.28" == %6.28
- char! type is now more explicitly supported, in the sense that single character strings will be loaded as char! so they are more efficient.
- object! type is now supported, so it becomes easier to emit TNetStrings with nested dictionaries and JSON data with nested objects. The converters can still (and need to) be compiled: they use the interpreter only very sparingly for objects support.
- The JSON converter now implements the full specification on json.org except escaped UTF-16 surrogate pairs. There is little reason for them to occur in JSON data.
The JSON converter is still smaller than the official R2 implementation. It's now larger than the R3 implementation, but has more features. It's still an order of magnitude smaller than most JSON implementations in other languages.
StoneDB is starting to take shape. I got the preliminary disk storage prototype finished today. I can't give factual speed benchmarks since for now I've got no time to do extensive testing... but it seems to be able to store at least 500000 nodes a second (@about 14MB/s), which is pretty decent for a prototype using default C disk writing functions and absolutely no regards for disk i/o profiling. this is even more acceptible considering its running on a lame notebook disk. (I should have a SSD after the holidays, so I'll be able to compare :-)
with the current architecture, I should be able to read any cell directly from disk so query set can be larger than physical RAM.
If all goes well, I should have persistent read/write access to the DB's file data done by the time I go to bed tonight ..... yay!
After that... cell linking which will require a different variable length dataset driver. This new one will allow perpetual appending without any need to copy memory :-)
I updated Red on Try REBOL with the latest Red fixes and the latest version of my JSON and TNetStrings converters:
I've released new version of my Redis protocol on https://github.com/rebolek/prot-redis . Now it's R3 module, adds some functions to make life easier (see documentation) and also few small bugfixes have been done.
NickA has agreed to allow his excellent "Learn Rebol" book to be published in "installments" in ODROID Magazine. Expect the first article to appear in the February 2014 issue which will be available at http://magazine.odroid.com .
This will help open Rebol up to a new community of enthusiasts.
Not quite ready for announce, but I've been working on a universal data manipulator type thingy which I've tentatively named 'munge. Reason I created it is I find myself using the following idiom quite often:
DESCRIPTION: Manipulate tabular values in blocks and delimited files. MUNGE is a function value.
ARGUMENTS: data -- (Type: block file url) size -- Size of each record (Type: integer)
REFINEMENTS: /header -- Ignore first row /where condition -- Expression(s) that can reference columns as A, B, etc (Type: block) /part columns -- Offset position(s) to retrieve (Type: integer block) /delete -- Delete matching rows (returns original block) /update action -- Update offset value pairs (returns original block) (Type: block) /unique -- Returns sorted unique records /list -- Return new-line records /only -- Return after matching a single row (ignored by /delete and /update) /save -- Write result to a delimited file file -- (Type: file) delimiter -- (Type: char)
and an example of some of the fun things you can already do with it:
test: copy ["Name" "Age" "Bob" 33 "Joe" 55 "Joe" 55] munge/header/where/list test 2 [B = 55] munge/header/unique/part test 2 1 ; this does not alter the block munge/header/unique test 2 ; this alters the block write/string %test.csv "Name,Age^/Bob,33^/Joe,55^/Joe,55" munge/header/where/list %test.csv 2 [B = "55"] ; file values are all string test: munge %test.csv 2 munge/header/update test 2 [2 [to integer! B]] ; update col 2 with val / block munge/header/delete/where test 2 [B <> 55] munge/part/save %test.csv 2 [1 2 2] %test2.csv #"," ; save with an additional column
That means they now include Red 0.4.1, and both PARSE and objects in the interpreter. Further, I added my JSON converter.
The */Red/red-core-message interpreter for your platform should be directly usable to write application servers for the Mongrel 2 webserver, using the 0MQ binding and the TNetStrings converter.
The */Red/red-core interpreter for your platform can for example be used to communicate with JSON servers such as Fossil and the Bitcoin daemon, using the cURL binding and the JSON converter.
The MSDOS/Red/red-base.exe interpreter can be used directly on Windows, without downloading any extra dependencies.
The Red development document I started is here https://github.com/iArnold/reddevdoc It needs a lot of extra info to be added. Please join the effort to make this a valuable intro in how the internals of Red are working together so more people can join and actually produce some code.
As all these Red docs and projects get started, is there an easy way to "register" or submit it to some centralized location yourself (perhaps with tags)
Since it's rather quiet here, I thought I'd post an announcement saying that I have a prototype of the next version of my camera software up and running on my camera hardware! Now, all processing is handled on the camera itself without the need for an external server, including the advanced motion detection. The camera can automatically transfer files for long term storage to one or more fileshare locations on the network. :-)
Oh, and almost all of the camera software is now running on R3 ARM and Red/System! I've removed most of the software that was written in other languages, like bash, except where absolutely necessary. :-)
In addition, the camera can now serve MJPEG images while simultaneously recording 1920x1080 30fps video. This allows the camera to act like a legacy IP camera so it can be used with software like iSpy, but still have even higher quality recordings to go along with the lower quality ones that legacy IP camera software can capture.
I still have some optimizations to do, but it looks very promising.
Since we had a server crash before x-mas, we had to move all our infrastructure to different services etc. Anyway, that was the cause why some documents of http://development.saphirion.com didn't work. The R3-GUI stuff is now fixed. Since our code is now located at beanstalkapp, and we read the docs and images directly from SVN and create the output, please give it a bit of time for long documents (especially layouts take time).
The Saphir stuff is not yet done. That's next. Expect it to be fixed in the next 24 hours.
The February 2014 issue of ODROID Magazine has an article on Rebol 3 running on Android, starting on Page 28, plus an edited version of the beginning of NickA's excellent LearnRebol tutorial. Check it out here: http://magazine.odroid.com
My cameras (Smoothcam) now serve H264 (via SMB) and MJPEG (via HTTP port 8080) simultaneously, and can also simultaneously serve individual JPEG stills through HTTP port 80. It can serve the H264 at one resolution and frame rate and the MJPEG stream and JPEGs at a different resolution and framerate. And I still have up to 35% of the CPU idle.
I just got word from HardKernel that the ODROID Magazine PDF for January 2014 has been downloaded ~8000 times. That's pretty good readership for a new magazine, I'd say. We're expecting a larger number than that for the February issue. The March issue is due out in three weeks. :-)
We have published our REBOL 3 View builds for Linux and Windows at http://atronixengineering.com/downloads.html. There is full Linux support for graphics and the clipboard as well as a number of bug fixes in the core interpreter.
This is the result of our collaboration with Saphirion since last summer. Our source code has also been pushed to the Saphirion repository to be tested and included in their own future releases.
Please note that we are preparing REBOL 3 for commercial use later this year, so we have removed all Alpha badging, cleaned up the splash screen and pointed the upgrade and demo functions to the Atronix website. The upgrade function is the best way to check for and download the latest Atronix release.
Please send comments and bug reports to firstname.lastname@example.org.
Announces here are now automatically cross-posted to the SO chat group via a RSS feed. And all web-public groups are now being mirrored on http://www.rebol.info/altme ...
Munge 1.0.5 - Minor speed improvements - Fixed minor merge bug - Fixed /part make block! bug - Removed string! as a data and save option - Added /order - Added sqlcmd /key - Added datestamp func - Documentation updated
I added the new stream! socket type to the Red 0MQ binding, for 0MQ 4:
I added HELP, WHAT and SOURCE from Ingo Hohmann to the interpreter consoles, in */Red/red-* I upgraded the 0MQ Windows binaries from 3.2.3 to 4.0.3. 0MQ 4 supports raw TCP (stream) sockets and a comprehensive encryption security framework. The Red binding doesn't have explicit support for it yet, but can already use the basic security features.
It has a new tree view mode for the files page. Some people seem to have a problem with the timeline being the default page for my repositories, so I changed the default page for all my repositories to the files tree, such as the above Red binaries and 0MQ binding repositories, and 18 others.
Since Red now has HELP and WHAT, I changed Try REBOL to execute the "Help" and "Available Words" buttons with Red instead of R3 (by default):
"Lest" is low entropy system for templating, HTML dialect similar to Henrik's HTML dialect, but with direct support for things like Markdown, Bootstrap, different Google services, etc. It's modular design make it easy to add more plugins to support different web technologies. It's still in early phases and some things are not finished yet, but it's already usable as it is.
http://lest.iluminat.cz/ has been updated with conditions. Everything can be replaced with condition now. Few examples: either why-not? div span "content" div either [now/time < 12:00] .am .pm "Hello" etc...
An Update on StoneDB. --------- I am FINALLY delivering the first prototype for review tomorrow. Its taken quite some time but its improved a lot, it now even has an API layer, which is the basis for the forthcomming query functionality. :-)
right now, I'm building it as a DLL, linking to it in R2 and am using memory buffers with parse for i/o with the DB. almost every part of the system is aimed at maximizing resources and reducing or practically eliminating memory fragmentation. Not all of the design is implemented, but what is done is extremely robust with a full stack error reporting mechanism, optional db function tracing log file (for very detailed support debugging, without debugger mode compilation).
Although only about 10% of the Server is implemented (in terms of having a robust multi-threaded multi-client "server") the storage and low-level api are already at v2. its all been re-implemented at least twice, sometimes even reviewed again.
I still can't say when I'll be opening up the source publicly. right now, I've got no time to handle any type of request or discussions on it, so there really is no point in wasting precious time on this.
this week and the next I will be adding the set theory query engine called AQUA . AQUA is an acronym for Accumulate & Qualify. basically a system which scrubs the relationships and data of any item, qualified by various properties where each phase can either accumulate more data or reduce it by qualifying it against many generic properties.
by supplying a simple chain of operations, you can traverse any dataset (in fact, several superimposed/remodeled versions of datasets) and try to find arbitrary and generic patterns within.
if you read through these you'll see that from an information system point of view they are closely related. I am not trying to support and describe my system using all of the (relatively strict) theory and some of the rather complex math lingo involved, but I am definitely reading this stuff over and over and it is influencing the design and philosophy of the entire system. for example, datasets are called domains, queries are perfomed using set operations, there will be possibility for some domains to be functional, so given an node ID, instead of storing a value, it calculates it on the fly, and you'll be able to build directed graphs of information even when using these computed datasets!
and the fun part of this is that your datasets need not be symmetric, but any sufficiently defined related sets of data will return a result. for example, any sales without a related date are automatically rejected from the above query. we could still get other data from those sales, if criteria is not related to dates.
all of this without knowing anything of the DB or how its supposed to be setup. If data is there, it browses it and extract datasets without requiring you to "search" thru gigabytes of information... it uses item associations directly for extremely fast navigation/extraction.
I've updated my binary downloads with the latest Red changes:
In particular, I added FreeBSD binaries. It's likely that some of the names and versions of shared libraries used by my bindings need to be adapted for FreeBSD, so I'm interested in reports of success or failure, and in updates to the names or versions of the library files on FreeBSD.
I tried to replace my simple CALL implementation by Bruno Anselme's new one, but his includes an incomplete copy of my C library binding, so they conflict and I can't use it in its current form.
FYI, I have upgraded my server to a safe openssl version, so CureCode and other web apps I am operating are protected for the infamous heartbleed vulnerability.
If you are running commercial apps and have used OpenSSL for the last two years, you should force people to use new SSL certificates if the site contains sensitive information, even if you have patched your server. the keys it contains may have already been breached and no one would know.
Thanks to Barry Walsh (draegtun), Github now should have significantly improved detection of Rebol code when using the .r suffix. For example, red/red is now more correctly identified to consist of ~99% Rebol code.
Further, the .reb suffix is now also supported to unambiguously identify (and thus: syntax highlight) Rebol code. For example:
Finally, some changes I pushed to Pygments a while ago seem to now be live on Github as well. That means that most Rebol files were syntax highlighting was previously somewhat broken should now be more properly highlighted. One case I can remember which was previously broken is prot-http.r from the R3 sources:
Nothing specifically Rebol or Red related in this month's issue of ODROID Magazine, but it is still an interesting read as it may give some of the Rebol/Red community ideas for improving on something presented using a different language.
Next month, however, I'm publishing an article on the ODROID Show serial/USB-controlled LCD screen - I wrote a dialect for controlling it with Rebol 3. If anyone wants the dialect before the issue comes out, PM me here.
Just thought I'd share that I've done a few releases of Glass and many of the slim libs in the last weeks. many goodie in there.
This year we did a BIL in SF, Canada, Tunisia (it was AMAZING), and now............................... Paris!
I submitted the pull request that includes serial code for Win32 and POSIX. This is originally based on Carl's posix serial code (from ReCode) and I have extended it to Windows. There is still a little cleanup to do, but it should work if you need to use it!
For those who are serious about it, I have a prototype of STONE DB which I have been given the green light to share to a few people.
This release is not open source yet, its a business thing, but I have been confirmed by the company partners that later this year we will be opening up the source. on my one year old lenovo ideapad y580 (core i7) I am averaging 250'000 item insertion and 250'000 links per second.
basically, the db is able to run a speeds which flood the iteration rate of REBOL.
PM me for a download link.
StoneDB now has a full DB index search. so you can search for any value in the db accross ALL non-infinite datasets.
fun fact is that the index is implemented using the db itself, its just another dataset which I am using as I would any other. (linking things to the index). so it doesn't require ANY extra code in the db back-end.
The June 2014 issue of ODROID Magazine is available for download! On page 6 is an article entitled "Programming Your ODROID-Show - Using the Rebol programming language to improve the hardware interface" by yours truly. It outlines a dialect I wrote to handle displaying information on the discrete display device called the ODROID-Show.
The ODROID-Show is a 2.2 inch display with 320x240 resolution and 64K colors controlled via USB through an Arduino microcontroller available through Hardkernel for $25. As the ODROID-Show works on any computer with a USB port, the Rebol dialect can also be used on any computer to display to it.
Lest https://github.com/rebolek/lest - HTML templating system - has been updated with structural logic (IF, EITHER, SWITCH, FOR). It's probably not Turing complete yet, but I guess it would be soon. Other improvements are for example /save refinenment for easier usage (write %file.html lest read %file.lest can be now replace with lest/save %file.lest) and numerous bugfixes.
great news maxim -> PM
I modified my Red bindings and codecs to work with the current Red development version, meaning an upgrade to Red 0.4.2 and beyond.
I'm experiencing regressions in the compiler and interpreter, though, so be careful
I put Try REBOL behind a new, more reliable proxy server:
I wrote a dialect for GPIO, specifically for the ODROID I/O Shield, that I am outlining in the August issue of ODROID Magazine. It handles configuring pins as input, output or PWM (pulse-width modulated). It could be easily modified to work with other GPIO systems, like the Raspberry Pi or other systems. Here is an example of the dialect:
gpio [ init out [p00 p07 p12] on ;Initializes pins p00, p07 and p12 as outputs wait .1 ;Waits .1 second off ;Turns off pins p00, p07 and p12 init in p16 ;Initializes pin p16 as input init pwm 1 .1 p01 ;Initializes pin p01 as PWM with a 1 second period and a .1 second pulse p12 on ;Turns pin p12 back on wait .5 ;Sleeps for .5 seconds p07 on ;Turns pin p07 back on wait 1 ;Sleeps for 1 second off ;Turns pin p07 back off deinit p01 ;Turns off pwm and deinitializes pin p01 init gpio in p01 ;Reinitializes pin p01 as a GPIO input read speed p01 ;Reads the value of pin p01 into system/gpio/vars/speed reset ;Deinitializes all GPIO and PWM pins ]
Other than loading the dialect, nothing else has to be done first. The dialect takes care of making sure the GPIO system is active, and activates it if it isn't already. The dialect was designed to decrease wordiness while being very easy to understand and follow. One way it does this is by keeping track of the context and applying any ambiguous commands to the most recent subject.
I published a few updates to my minimalist "shttpd" HTTP server implementation for Rebol 3. Most notably, a workaround was added to properly send reponses >32'000 bytes (thanks to Bo and Boleslav for prodding me about this).
The full list of changes:
Add a few select MIME types Switch default MIME type to application/octet-stream Use last "."-separated path component as extension Trim indentation from the error template Send content-length header Manually send body data in 32'000 byte chunks
I added ARM-HF builds for Linux in the "RPi" target. folder. Because Linux distributions with the old 0MQ 2 versions, particularly the previous Ubuntu Long Term Support, are now being phased out, I upgraded the builds for other systems than Windows to 0MQ 3/4. (0MQ 4 was already included for Windows.)
The August 2014 issue of ODROID Magazine is available at http://magazine.odroid.com. In this issue, I have an article on how I go about writing dialects, using the example of a GPIO dialect (General Purpose Input/Output) I created for the ODROID IO Shield using Rebol 3. It could be easily modified to work with other GPIO applications, like the Raspberry Pi. In retrospect, I would change some of the dialect decisions I made and some of the implementation. The entire dialect and article was designed and written in one day, so I didn't have a whole lot of testing before publication. That's my excuse, and I'm sticking with it. :-)
The September 2014 issue of ODROID Magazine is available at http://magazine.odroid.com. I didn't have time to submit an article for this issue. (However, I am wondering if anyone noticed that the "Meet an ODROIDian" article last month was about me as nobody mentioned it.) Here's the table of contents for this month's edition:
6 Playing Sega Games in HD 1080p: A Blast from the Past 7 Linux Gaming: Playing Native Games on the ODROID - Part 1 9 Building an All-In-One DIY Laptop: Take Your U3 Anywhere 10 Bash Basics: Shebangs and Shebangs 12 Installing Freedomotic: A Building Automation Framework 13 Installing WICD: A Network Connection Manager 14 3Dponics: An Open Source ODROID-Powered Gardening System 18 Wall-E: Building Your Own Robot At Home - Part 1 23 Weather Forecast on the Desktop: What Chance Is There to Catch Fish Next Weekend? 25 Digging (Into) the ODROID-SHOW: Part 2 - Making Connections 28 Headless 10-Node ODROID-U3 Cluster: The Ultimate Affordable Home Supercomputer 30 Android Development: Inside the Android APK 33 Meet an ODROIDian: Tobias Schaaf, Linux Ninja and ODROID Enthusiast
Great, Bo. I'll check out last months for that article....
Thought that some of you might want to throw in your $0.02.
The AltME feed is reportedly back up again on SO Chat.
Shixin Zeng announced the availability of an (open source) encapper for R3 on Stack Overflow chat. Here's the original text from Shixin:
"Anybody interested in bundling the REBOL script into an executable? I have worked out something that can build such an executable for both Linux and Windows. The change is at: https://github.com/zsx/r3/commit/c4a478b9"
From the referenced commit message:
"Add an encapper for Windows and Linux The way it works is: 1. embedded the script into the executable file 2. the interpreter checks for the existence of the embedded script 3. if it finds it, runs it and ignores "--do" and script passed in from the command line 4. if not, act as a regular interpreter
The encapper on linux is just a wrapper around objcopy, wihch adds a section with the script to the executable
On Windows, the script is added as a resource, and some windows APIs are used to retrieve the information."
I'm also happy to report, that some initial testing shows that Atronix's R3 encapper works on Linux quite nicely.
I managed to build a fully statically-linked 64-bit R3/Core binary with a "Hello, World!" script encapped that comes out at ~800K in size.
Being fully statically-linked (using the musl C library) means that it's completely standalone: no libc, libm, libdl, etc depencies (those who have encountered libstdc++ dependency troubles with R2 might appreciate that). The only dependency left is a Linux kernel (with a compatible interface). So such binaries should be able to run unaltered for many, many years to come.
On the R3 side, this binary includes a 64-bit core, HTTPS support, a full CALL with I/O redirection, the FFI (/Library), etc. Pity that the encapped "Hello, World!" script doesn't make any use of all that :)
Not specifically Rebol-related, but we hope to promote Rebol through our new channel, http://ameriDroid.com .
ameriDroid.com is now the official US distributor for Hardkernel's entire ODROID product line. ODROIDs are some of the most powerful single-board computers available in the world, running primarily Linux and Android. Check out the site and let us know what you think!
I'm happy to announce that rebolsource.net now also provides a FreeBSD x86 build from mainline R3:
Feedback: WOOOO YEAH! We need a simple way to send, receive, and visually manipulate native Rebol data series (a list and *Grid* widgets, especially). Tab panels, accordians, and other page layout options would make this more than just cool. I'm ridiculously excited about the potential this little thing :)
Where can we get it?. I'll happily donate and write examples.
- SQL Server 2012 fix (blank lines embedded in output) - Refactored load-dsv based on csv-tools.r - Added /max and /min - Added /having - Fixed /save to handle empty? buffer - load-dsv now handles xls variants (e.g. xlsx, xlsm, xml, ods) - Fixed bug with part/where/unique - Added /compact - Added console null print protection prior to all calls - Added read-pdf (requires pdf2text) - Added read-xls (requires Excel)
I have updated the development version R3 at http://atronixengineering.com/downloads.html, all of them should now have a CALL with I/O redirection and be able to do async CALL. The implementation is still experimental, please give it a try and report any bugs as you see. The only difference between Linux/Windows version is that CALL in windows doesn't accept a block! or file! for its argument, it only accepts a string as the command.
Compatibility patches to-error does not work in R3 remove-each R3 returns integer select R2 /skip returns block unique /skip broken Minor changes to work with R3 read (R3 returns a binary) delete/any (not supported in R3) find/any (not working in R3) read/lines (not working in R3) call/show (not required or supported in R3) call/shell (required in R3 for *.vbs) Removed /unique Added column name support Added /headings Added /save none target to return lines Merged /having into /group worksheet changes Removed columns argument Removed /widths and /footer refinements Added spec argument Added support for date and auto cell types
If you like to be bleeding edge, you can now access our web sites, including Try REBOL and our Red and REBOL sites and Fossil repositories, through the HTTP 2 protocol, the successor to HTTP 1.1 and Google's SPDY protocol.
You may have to enable HTTP 2 in your browser or other client.
Ren Garden has been built for Linux, Mac, and Windows, and variously for 32-bit and 64-bit architectures. Though it's linked up with Rebol for now, it was designed to be linked against Red also. The conditional build is set up, so it's just work that someone needs to go in and do.
Where these projects can (or will) go depends on community support and interest. So far response has been very encouraging...but to push this from demo to 'product' will require more hands and heads than just mine!"
Here are the features Ren Garden brings over that built-in Rebol REPL:
Undo - This allows you to clear clutter in the REPL by undoing input and output.
Variable/Expression Watches - A great aid for debugging, watches show the current value of words or expressions. Anything being watched can be duplicated, and frozen to help in comparing values over time. Watches can be renamed to give them memorable names.
Multiple Buffers - Any number of buffers can be created and accessed using a tabbed interface. The tabs are nameable to make it easy to distinguish them.
Separate Contexts Across Tabs - Right now, each tab is given its own context (essentially a new user context) when it's created so there don't need to be multiple Rebol/Red runtimes running at once to achieve a certain degree of isolation. Though the creation of a new context or complete runtime per tab is not currently configurable, it could be made so.
Shell Integration - This feature makes it easy to interact with or automate the OS shell being used from a Rebol/Red session.
Auto-Completion - Basic auto-completion is available.
Beginnings of Buffer "API" - There is a demonstration of buffer cursor and selection control which could be used to provide smart snippets or to enable interactive tutorials to be created.
Ren Garden also includes some of the Rebol language experiments that Hostilefork has been talking about to allow people to try them out and give feedback on.
The projects can currently be found at the following link. Note that Ren Garden will be pulled out of the RenCpp source tree not too far down the road.
I did watch the video. For those not familiar with C, I am wondering if someone could clarify the terminology, specifically, "a C++ binding for Rebol or Red." Does that mean a way for a C program to call a REBOL program? Or a REBOL program to call a C program? Or something else? Thank you.
Replaced to-error with cause-error Replaced func with funct Added execute function Added MS Access support to execute Added SQLite support to execute Added /only (was /unique in 1.06) Added spec none! support Added /save none! support Fixed /merge bug Fixed sqlcmd /headings/key bug Added cols? function Added rows? function Added sheets? function Fixed to work with R3 Alpha (rebolsource.net) Added load-dsv /blocks Fixed delete/where (missing implied all) Added unzip function
Added call compatibility function for R3 Alpha Added /all support to read-xls Added /part to load-dsv Re-factored VBS calls Added fields? function Added associative array support (index, lookup, assign) Added unique undex support (index/direct, retrieve)
I know that the Try-It Editor allows infinite loops and other things that can crash R3. However, I have disabled all the damaging functions like 'call, 'write, etc. so at least there's that. I'll soon be adding a timer to kill the Try-It process if it runs longer than a few seconds.
Not specifically Rebol/Red related, but I wanted to let you all know that ameriDroid.com is now an official distributor for Banana Pi, Banana Pro, and Raspberry Pi 2, in addition to exclusive N. American distributor for Hardkernel/ODROID. We also carry a full line of accessories and peripherals. I hope you give us a chance if you need any of the systems that we now carry!
Refactored main munge function removed /flip refinement removed %.xml, %.xlsx and none! as valid /save targets added /group 'avg removed implied Excel column names /update can reference column names without /where (e.g. munge/update blk 2 ['c1 [c2 * 2]]) split /merge off into a separate function split rowid referencing off into separate functions (pick-cell, poke-cell, copy-row, remove-row) big speed improvements (about 40%-60% faster for most /where and/or /part operations) Added a lot of support functions No longer redefine 'select (new select-skip function added) No longer redefine 'unique (new unique-skip function added) No longer redefine 'call for R3 Alpha (use R2 or Atronix R3 instead)
Ren/C is an interim fork of the open sourced Rebol codebase. It has many goals:
To create a library form of the interpreter, which is focused on providing the full spectrum of Rebol's internals to other projects. This is to open up possibilities for linking to new IDEs, debuggers, and consoles (such as Ren Garden). It will also be the basis for language bindings like Ren/C++.
To un-fork the divergence between the community build of Rebol based on the 12-Dec-2012 open-sourced code and the codebase built on pre-open-source code at Saphirion, later maintained by Atronix Engineering.
To accelerate the process by which consensus features and fixes are integrated into the language. Though debates continue over features in Rebol and Red Chat, many changes are ready and agreed upon--some submitted as patches. Yet they haven't had a focal location where they can be launched and people get new builds.
To integrate the test suite into the build, and make the bar obvious that contributions must meet by keeping it at zero errors. The Valgrind and Address Sanitizer tools are being integrated into the build and test process, with modifications to the code to prevent masking bugs.
To provide an option for stable open-source bootstrapping to be used by the Red Language, which is currently bootstrapped from closed-source Rebol2. (Red's roadmap goal is to move directly to a self-hosting state from the Rebol2 codebase. This may be a poorer option than moving to an improved Rebol3 as an interim step.)
The Atronix/Saphirion build diverged from Rebol at a point in time prior to its release as an Apache-licensed open-source project. Their build had a graphical user interface and several other additional features, but was only available for Windows and Linux.
Ren/C split out a "Core" build from the Atronix/Saphirion branch, which runs as a console process and does not require GUI support. It was then audited to build under strict ANSI C89, C99, and C11. It also added the option to build as strict ISO C++98, C++11, and C++14. The goal is to take advantage of stronger type-checking and metaprogramming, while still retaining the ability to do a complete build on very old compilers when __cplusplus is not defined.
Consequently, Ren/C brings all the non-GUI features added by Atronix and Saphirion to core builds for other systems (Mac 32-bit and 64-bit, HaikuOS, Raspberry Pi, etc.) It also allows users who are not interested in the GUI to use lighter builds on Windows and Linux.
Besides building under these ranges of languages, Ren/C can do so under both GCC and Clang with zero warnings (with as strict warnings as reasonable). Significant changes are needed to do this, which are being given heavy thought on how to make things simpler, clearer, and better-checked. Across the board the code is more readable than what came in, with notable simplifications and improved checks.
(Note: Ultimately the goal is that Ren/C will not be something Rebol end-users will be aware of, but a library facing those building software that links to the evaluator. Hence systems like Rebol and Ren Garden would be branded as interfaces and consoles using the core interpreter, and Ren/C would contain no 'main.c'. Getting to that point will take a while, and in the meantime Ren/C produces a traditional Rebol executable as part of its build process.) Features
New features available in Ren/C's console builds vs. the open-sourced Rebol codebase of 12-Dec-2012 are:
HTTPS support as a protocol written in Rebol code, empowered by underlying cryptography libraries incorporated the C code.
An implementation of LIBRARY!, which allows Rebol to load a DLL or shared library and then directly call functions in it. This is accomplished with the "FFI" (Foreign Function Interface) and new data types for representing C-level constructs like ROUTINE! and STRUCT!.
Note: Building Ren/C with FFI currently requires additional steps or package, installation, as the FFI library has not been extracted into code following Rebol's build process.
CALL with /INPUT /OUTPUT /ERROR
UDP Network Scheme
Ability to make use of native ("__builtin") 64-bit math, if it is available
(Additionally there is serial port support on Linux and Windows.) Platforms
As of 16-Jul-2015, Ren/C has been verified as reaching the goal of building across the standards-compliant spectrum of C or C++ without warnings on these desktop platforms:
Linux 32-bit, libcc 2.11 (OS_ID=0.3.04) Linux 64-bit (OS_ID=0.4.40) Windows 32-bit (OS_ID=0.3.01) Windows 64-bit (OS_ID=0.3.02) OS/X 32-bit (OS_ID=0.2.05) OS/X 64-bit (OS_ID=0.2.40)
It has additionally been built for:
ARM Linux on Raspberry Pi (OS_ID=0.4.21) OS/X PowerPC (OS_ID=0.2.04) HaikuOS (OS_ID=0.5.75) SyllableOS Desktop(OS_ID=0.14.01)
The tools above are an older precursor to CrossUI, but I like them because they're free and open source, really light weight, and not only the created apps, but also the entire development system run virtually anywhere - even IE6, Firefox 1.5, and in just about any mobile browser. It's really simple to learn to use, and extremely productive, at least for the needs I've encountered. You can package the GUIs for mobile platforms using Phonegap build (and desktop with Node Webkit - although I don't see as likely a need for that). You can run the visual builder from any Apache install on any OS - I've done it locally on Windows, on Android with KSWeb server, on shared hosting Lunarpages accounts (which I've been able to use even on the old Android phone). It's nice to be able to quickly and easily create full featured front ends on any device, which run on any device, and which connect easily with Rebol server scripts that run just about anywhere too, including shared hosting. The whole system is really simple to use, and exhaustively documented.
The video is 1.3 hours long - very sorry for the length - I did it to prepare for making a written tutorial which will be more succinct. All the files used in the video are here:
That's a fun thing to see. Some decades ago, when Burroughs still was a computer company, one could, in a COBOL source program for their Medium Systems line, type the declarative ENTER SYMBOLIC and then start coding assembler language. Second-generation code embedded in a third-generation program. Now with the C-in-REBOL example, we see third-generation code embedded in a somewhat-fourth-generation program.
Rewrote to be R3 specific R3 optimisations (20% faster than R2) Removed lots of R2 compatibility code Excel functions refactored to use Powershell/OLEDB (so MS Office is no longer required) load-excel function added
AltJSON v0.3.5 for Rebol 3 (including current Ren/C version)
- Supports easier creation of JSON objects with set-word/value pairs in blocks, get-word/path lookup. - Uses MAP! to correspond with incoming JSON objects. - /FLAT refinement uses blocks for incoming JSON objects to allow for case-sensitive keys, parsing and null values.
- Introduces some of the changes from the Rebol 3 releases: - Switch to RFC 3339 formatting of dates, - Resolves Get-Words and Get-Paths, - Recognises Set-Word/Value pairs as JSON Objects, - Limits the format of JSON object keys, uses ^ escaping on words that almost conform (use /Flat for more flexibility)
Individual posts to this group will now have their own page. This page will be referenced to by the feed for this group (http://rebol.info/feeds/announce.feed). Colours will be respected, links detected and full names included. Each page will link to the reflector page for this group.
Another web site of documentation related to REBOL 2 has come into being at http://www.cobolrebol.com/. This is some learning aids for a very specific and small audience, and has been put on the internet in case it is helpful to anyone else.
From renewed rebol.com frontpage: rebol.net is back online (checked it too) (Thank you Carl!)
We (i.e. Atronix Engineering) have published a Rebol 3 function reference on our wiki. It is specific to our build and automatically generated using the interpreter and a meta data file for things like examples, cross reference and comments. Hopefully, this level of automation will mean that it stays updated. All of the examples have been tested.
The wiki is currently read-only to the public. Feedback is welcome.
Goldie, the robot, does the entire presentation herself in one take.
added APPEND-COLUMN, FETCH, LATIN1-TO-UTF8, ROWS?, TO-SQL-DATE updated LIKE, OLEDB?, SORT-ALL (added /reverse) renamed EXCEL-SHEETS? to SHEETS? removed LOOKUP replaced EXECUTE with dedicated SQLCMD and SQLITE functions replaced WORKSHEET with WRITE-EXCEL (about 10x faster) removed all dependencies on VBS and temporary file creation fixed minor Excel column handling limitations excel functions now work with accdb use of deline/lines to speed up line parsing
Arnold, all the forks of rebol/rebol are the same repository, including ren/c they cannot delete rebol/rebol.
Github won't remove it anyways. why would they ?
Another REBOL-related document has come into existence on the internet. This is documentation, for a very specific audience, about how one might use REBOL for working with data outside REBOL's "natural habitat," in the land of fixed-format data files. It tries to explain which REBOL functions are most useful in that area, and to provide some helpful code modules. The document is located at: http://www.cobolrebol.com/pages/documentation/FixedAndCSVFiles.html
New version of The World Programming Language is out. This is a major alpha release with lots of new stuff incl. non-blocking networking by implementing libevent..
OLEDB error handling improved rewrote append-column and added /dup refinement added merge/default refinement load-dsv parse logic fix to handle utf8 Replaced to-sql-date with to-rebol-date and to-string-date rows? and first-line faster load-dsv/delimit renamed /with load-dsv/split now a separate split-dsv func added unarchive can update via conditon (e.g. [all [find c2 "@" c2: first parse c2 "@"]]) munge now accepts vector! alphanums?, alphas? and digits? now also accept binary! digits? also checks for not empty replaced like with like and glob added write-excel/filter added spec? function sqlcmd now does a SET ANSI_WARNINGS OFF added /string to load-excel added /hdr to load-excel added remove-column added to-string-time added binary-file? added spec?/as-is fixed obscure group bug with to-block handle presence of u160 re-added formula support to write-excel, e.g. write-excel/filter %test.xlsx [ "Test" ["Response" "Count" "Yes" 10 "No" 5 "" "=SUM(B2:B3)"] [20 10] ]
Red's github repo just passed 1000 stars bar, not a big deal compared to mainstream languages, but still significant enough to mention. This is the first time an open source project from Rebol's ecosystem reaches that level. This means that we are on the right track, and Red is gaining more momemtum with each new release. Hope this will motivate more people to help us continue pushing it higher and higher. ;-)
Approaching 300 members on the Gitter channel now too.
1000 stars is great! Hopefully half of them are ladies! 8-)
I've moved my backup of DocBase from an unrelated repo in my personal account to a dedicated repo within the Revault group. I've been tidying it on an as-I-notice-things basis as I use it for reference. Let me know if you have any trouble using it.
For those who don't know, Pieter Hintjens, of ZeroMQ and Xitami fame, passed away last Tuesday.
Sad, I read his "protocol to die" a few weeks ago.
Major rewrite: - Same script works with both R3 and Red - Block format has changed from flat (e.g. [a 1 b 2]) to blocked ([[a 1][b 2]]) - Single context with functions that can be selectively exported to global context - Functions simplified and non-core options split off (e.g. munge/list -> list, munge/save -> write-dsv) - General speed and efficiency improvements - Munge function supports 1st column binary search - OLEDB [Windows only] functions (e.g. load-excel) work with both 64bit R3 and 32bit Red - Some missing Red functions (e.g. delete, delete-dir) implemented via CALL [Windows only]
This is a preview release: - to-rebol-date and to-string-date do not work fully with Red (as it currently lacks date! support) - some functions have not yet been enhanced to work on non-Windows systems - write-excel requires 7-Zip to be installed and will only work on Windows - The now* function is available when run from Red and provides many [date related] options missing from now - Basic documentation is available at http://dobeash.com/files/munge3.html
I have claimed the name R3N on github and set it up as an organisation. The purpose of this 'organisation' is to overcome the status quo on the rebol/rebol repository this community has been put in. In r3n the github repo of the REN-C repository has been cloned. Graham Chiu and Giulio Lunati have been added as owners of the organisation. Work on collecting R3 stuff has begun. Making a start on https://github.com/r3n
Documenting the Red development project: https://github.com/reddoc/develop/wiki This project is to help interested developers to quickly get on speed in the codebase. All interested are invited to add to this project,so for example programmers with Linux background can help on the GTK3 branch.
Minor fixes: - added /preserve refinement for load-dsv - now support added for Red - removed dependency on 7zip - read-pdf now works on Mac / *nix - write-excel now works on Mac / *nix
I started a place to collect Red scripts that the community members might like to share with others, a bit like rebol.org it is https://github.com/freered/Code Together with it comes a place to collect valuable documentation for cooperating on the yet unexplored territories like 64 bit and GC. Also there is a fork to host cooperation of community members that want to work on these things without disturbing the official branch. https://github.com/freered All comes with a own gitter chat room https://gitter.im/free-red
Google gives new users on their cloud service USD300 credit which is enough to run a couple of VMs for a year. Here's a short HOWTO on how to setup CGI on a Google Compute instance. Any errors are my own and fixes/suggestions appreciated. https://github.com/r3n/renclib/wiki/CGI-on-Google-Compute
AWS is free for 1 year too.
I dont know if it s the right place but i m searching after any trial of a integrated voting machine based on the blockchaine (any type of : ethereum or bitcoin). Is there any open sourced code implemented in any version of Rebol ?
Not the right place .. this is announce. Use announce reply to anothe channel. and there is no blockchain code that is open for Rebol*
send email@example.com "Come back home, all is forgiven!"
After the recent outage at rebol.org and the subsequent postiing of the script library on github, I downloaded my own personal copy of the library out of fear of another disruption. I have that copy tucked away on my computer, but I also put it on my little web site in case such a backup would be useful to others. If that was improper in any way I would be happy to delete it, and would not be offended. The location is here: http://cobolrebol.com/pages/RebolOrgscripts/RebolOrgScript-index.html
Red release 0.6.3 is ready. Support of macOS. Next best in this release must be the test backend allowing automated testing of GUI!
Minor changes: - Simplified code now that Red 0.6.3 supports date! and delete - All test cases (200) now work with both R3 and Red 0.6.3 - Basic documentation available at http://www.dobeash.com/files/munge3.html
Minor changes: - Refactor to work with Rebol2 and Ren-C - All test cases (200+) now work with R2, R3, Ren-C and Red - Windows-specific features disabled on non-windows platforms - Updated http://www.dobeash.com/files/munge3.html to cover Excel, SQLCMD, SQLite, PDF dependencies
Implements a (largely) HTML5 compliant markup parser in Red and Ren-C and LOAD-MARKUP function using said parser. This is an experimental hybrid script and uses Red macros in a way that might conflict with other Red code--use with caution.
LOAD-MARKUP returns a block akin to Rebol 2's LOAD/MARKUP except that attributes are contained in a MAP! following the associated tag and self-closing tags are indicated by a </> closing tag following the tag or attributes (if present).
To do: tree-based model and DOM (likely similar to AltXML). Some work to be done here--the HTML5 rules for tree-building are labyrinthine.
I've added the tree constructor per the HTML5 spec and added a few more standard error codes (with source positions). There's a few kinks--while table nesting seems to be working and for the most part misnested formatting is resolved properly, there are still failures on the odd test or two*.
It's all still a bit slow for my liking--the parser is set up near exactly the way the spec suggests which in no way works to Rebol's (or Red's) parsing strength--pretty sure that's ripe for optimization**. Also still not sure about my choice of linked lists for trees either--there's a real gain in fidelity (good for editing when the time comes), but I'm not convinced there's not an insurmountable hit on navigation speed***.
Am still working in a Red/Ren-C compatible way (has it's challenges but is great for rooting out bugs!). In Red you can just DO the raw github url. In Ren-C, you can: import <markup>
LOAD-HTML will return a document node from a given string, MARKUP-AS-BLOCK will convert that into a more familiar block hierarchy. If you get an actual Rebol/Red error, let me know--shouldn't happen.
*I feel as if I've read the 'Adoption Agency' algorithm 30 times over and still don't understand how it's supposed to work **my only reservation here is while it does slow things down, it would work well with an incremental byte stream, say from a network source, where that might not be as noticeable ***if anyone has any thoughts on the efficacy of linked lists, would be interested to hear--have two versions: one with maps (no key/value clashes) and one with blocks (does seem faster)
AltJSON v0.3.6.1 for Red
JSON decoder/encoder for Red--features copied from Rebol 2 version.
- Converts object (map) keys to words where compatible, strings otherwise - Decodes large integers as issues (for e.g. Twitter IDs) - Encodes dates as RFC3339 - Support for surrogate pairs (decoding/encoding characters higher than the BMP as ascii) - 'Flat' mode for parsing, duplicate keys - Contracts Flickr's '_content' objects
Located and rebuilt the articles from Rebol Forces. For historical and informational reasons mainly.
Lately at work I have had to do much dissecting and assembling of spreadsheets, as well as cleaning up things accumulated over the years. Learning from previous experience, any time I find any code that could possibly by used in the future, I try to encapsulate it into a module or function. Then I produce a version sanitized of any employer-specific data and put it on a little web site in case it might be of use to others. With today's batch of uploads, I have hit the small milestone of 100 little programs. Most are extremely simple, and extremely specialized. It is sort of like the rebol-dot-org script library but much simpler, maybe more useful to beginners. It does show, like Nick's tutorials show, the value of being able to program one's own computer. The scripts are in the "free stuff" area of http://cobolrebol.com/
It's a QT app which runs a QTextEdit widget as a console and links to the Rebol evaluator, and the C++ API. You can setup watch points to see changes in values in your code for debugging purposes. So, you could run this when debugging your r3/ren-c code.
renc support read-pdf /lines from read-string (use deline/lines instead) /compact from load-dsv /preserve from load-dsv
enzero unarchive archive
split-line bug write-dsv bug load-dsv bug to-string-time now returns "HH:MM:SS" put returns value latin1-to-utf8 handles split strings correctly load-dsv/part/flat formats data correctly
Added /map to sqlcmd Added /flat to load-dsv, load-excel and load-fixed load-fixed now auto detects widths read-string with large files about 10-15x faster load-dsv about 3x faster rows? faster Added /flat to list write-excel now uses archive instead of 7z
build, platform and target moved from global context to settings append-column ascii-file? average-of call-oledb delta load-excel (replaced with oledb) map-source oledb-file? order remove-column split-line sum-of to-rebol-date to-rebol-time
crc32 excel? load-xml (uses unarchive) oledb (refactored from load-excel) settings literal column support to load-dsv, load-xml and munge (e.g. load-dsv/part file [1 "No"]) named column support to load-dsv, load-xml and munge (e.g. munge/where next [["Name" "Age"]["Sam" 50]] [&Age > 30])
munge/update refinement removed (use munge/where blk [row/1: 0]) munge/delete is now the same as munge/delete/where R2 support in archive and unarchive (requires /View) Excel support (using unarchive) added to cols?, fields?, rows? and sheets? replace-deep replaces multiple values in a single pass to-string-date and to-string-time now support Excel XML date (e.g. "41000") and time (e.g. ("0.33") formats list uses settings/console (default TRUE) to control new-line behaviour load-dsv (plus oledb, sqlcmd and sqlite) and load-xml use settings/as-is (default TRUE) to control trim/lines behaviour read-string optimized read-binary lost its /seek refinement added load-dsv/csv
Several parse copy words that escaped to the global context Unarchive bug
I will be hosting a Rebol Developers Conference in Philadelphia this summer (July 6th-7th). I've a great location lined up just around the corner from the Independence Mall (across the street from Ben Franklin's Post Office) and have some great talks lined up (more on this to follow--some epic projects: some you'll know about, one or two you won't).
I'll have registration up shortly, for now I've a wee questionnaire to get an idea of what kind of interest there is. Aside from the featured talks, I'd like to now issue a call for participation for anyone that would like to talk about their Rebol, Red or Ren-C projects. We'll have streaming and will record the talks for posterity.
If you're in the northeastern US, Philly is very accessible (great rail/bus access) about 90mins from NYC and DC. If not, it's a culturally rich city that is very much worth a visit even without a Rebol conference as an excuse, and we'll be here for July 4th at the very location where July 4th became a thing. Our hosts will provide co-working space on July 5th for any conference participant arriving from out of town.
Updated web site for the Rebol  developer conference. Includes some info on the talks, speakers and location and a link to the event registration. Be sure to register as soon as you can so I can get a close-to-accurate head count ahead of time.
Also on the registration site, I'm accepting donations to help offset some of the costs (it's unlikely registrations alone will cover everything--I've deliberately kept the registration fee down to encourage wider participation).
At this time, there is help available if accommodation is still needed. Just drop me a line.
Only a week away now, hope to see you there!
A huge thank you to everyone that attended/contributed to Rebol  here in sunny Philadelphia*. It was a blast and there were some monumental presentations. I will update the conference site shortly with links and videos as they become available.
If you did contribute and would attach some links to your talks, please email them to me.
*it is always except for that moment a tornado warning interrupted Ed's talk
I just saw a Rebol conference took place this summer, it's great! Are the videos of the conference available online? I was not able to find them.
Thanks to Brian for organising and editing the recordings!
I suppose I am showing my age by not being able to figure out how to use github, but because of my age I don't care. For the past few years I have been saving any bits of code I have written that might be of use to others, and storing them in the "free stuff" area of http://cobolrebol.com/ where others can download them if they find them helpful. In a few months I will be entering the Age Of Fixed Income and will have less access to reasons to write REBOL code. Since my collection will become more static (as will I), I have copied the useful stuff to a github account called "cobolrebol" where they will remain in perpetuity, to be stumbled upon by some future programmer who will wonder, "What is all this stuff?" REBOL is dead. Long live REBOL.
SWhite - I am 1.5 decades away (at the earliest) from retirement. I've been pondering what to do when I do retire. I think I'll put more time into my innumerable hobbies, one of which is programming. Even though I program mostly for work, there are some home-based projects I'd like to do as well.
I also haven't put in the effort to figure out github. I'd rather build my own system that is as simple as possible and meets my needs.
One of the presenters at the aforementioned Rebol  Talks was a Brian Otto who showed his UI Builder. It's quite cool. https://youtu.be/mjVtXfdTZIQ (direct link). If you're into WASM, check it out. Brian Dickens did a great editing job on these videos, by the way. Very watchable.
Not sure how many here know about NLPP, which is a Non-Linear Performance Pricing tool written in REBOL. We just got a new partner. You may have heard of them:
Still a lot of work to be done. Anyway.. the system is not meant to be used only on Rebol... I was using some its predecessors to build other projects too... but Rebol is a little bit different with its high use of generated code.