minimal implementation of a PassThrough stream
fs read and write streams based on minipass
fs read and write streams based on minipass
Slice another stream via a transform stream
Filesystem-buffered, passthrough stream that buffers indefinitely rather than propagate backpressure from downstream consumers.
An iteration of the Node.js core streams with a series of improvements
Read/write IEEE754 floating point numbers from/to a Buffer or array-like object
smart-buffer is a Buffer wrapper that adds automatic read & write offset tracking, string operations, data insertions, and more.
Access the system clipboard (copy/paste)
Read and parse a YAML file
The Node.js bindings of the humanfs library.
read(1) for node programs
Bytes go in, but they don't come out (when muted).
Standard input/output streams for Bare
Contains parsers and serializers for ASN.1 (currently BER only)
Read and write a growable buffer as a stream
[![npm version][npm-version-src]][npm-version-href] [![npm version][codecov-src]][codecov-href] [![Master Workflow][workflow-src]][workflow-href] [![Known Vulnerabilities][snyk-src]][snyk-href] [![Conventional Commits][conventional-src]][conventional-href
Streams for reading/writing messages
read and write binary structures and data types
Read/write IEEE754 floating point numbers from/to a Buffer or array-like object
File edition helpers working on top of mem-fs
create a pipeline of streams using Minipass
fonts (ttf, woff, woff2, eot, svg, otf) parse, write, transform, glyph adjust.
A simple pipe for buffers
I/O pipeline construction framework. Allows to construct data processing pipelines in a manner of UNIX shell pipes. Implemented features: string splitting/merging, IO or local file reading/writing, FTP/SFTP file reading/writing, digest computing, Gzip/Zlib (de)compression, Zstd (de)compression, symmetric cipher (de,en)cryption, random data generation.
Fire and Forget replaces the need to write resque tasks or delayed jobs to fire off web requests (usually notification webhooks or a anti-spam service, like defensio or akismet). A single worker reads and executes web requests from a blocking named pipe, while clients queue up them up in a non blocking manner. It uses typhoeus internally to execute the web requests for maximum speed.
Log2json lets you read, filter and send logs as JSON objects via Unix pipes. It is inspired by Logstash, and is meant to be compatible with it at the JSON event/record level so that it can easily work with Kibana. Reading logs is done via a shell script(eg, `tail`) running in its own process. You then configure(see the `syslog2json` or the `nginxlog2json` script for examples) and run your filters in Ruby using the `Log2Json` module and its contained helper classes. `Log2Json` reads from stdin the logs(one log record per line), parses the log lines into JSON records, and then serializes and writes the records to stdout, which then can be piped to another process for processing or sending it to somewhere else. Currently, Log2json ships with a `tail-log` script that can be run as the input process. It's the same as using the Linux `tail` utility with the `-v -F` options except that it also tracks the positions(as the numbers of lines read from the beginning of the files) in a few files in the file system so that if the input process is interrupted, it can continue reading from where it left off next time if the files had been followed. This feature is similar to the sincedb feature in Logstash's file input. Note: If you don't need the tracking feature(ie, you are fine with always tailling from the end of file with `-v -F -n0`), then you can just use the `tail` utility that comes with your Linux distribution.(Or more specifically, the `tail` from the GNU coreutils). Other versions of the `tail` utility may also work, but are not tested. The input protocol expected by Log2json is very simple and documented in the source code. ** The `tail-log` script uses a patched version of `tail` from the GNU coreutils package. A binary of the `tail` utility compiled for Ubuntu 12.04 LTS is included with the Log2json gem. If the binary doesn't work for your distribution, then you'll need to get GNU coreutils-8.13, apply the patch(it can be found in the src/ directory of the installed gem), and then replace the bin/tail binary in the directory of the installed gem with your version of the binary. ** P.S. If you know of a way to configure and compile ONLY the tail program in coreutils, please let me know! The reason I'm not building tail post gem installation is that it takes too long to configure && make because that actually builds every utilties in coreutils. For shipping logs to Redis, there's the `lines2redis` script that can be used as the output process in the pipe. For shipping logs from Redis to ElasticSearch, Log2json provides a `redis2es` script. Finally here's an example of Log2json in action: From a client machine: tail-log /var/log/{sys,mail}log /var/log/{kern,auth}.log | syslog2json | queue=jsonlogs \ flush_size=20 \ flush_interval=30 \ lines2redis host.to.redis.server 6379 0 # use redis DB 0 On the Redis server: redis_queue=jsonlogs redis2es host.to.es.server Resources that help writing log2json filters: - look at log2json.rb source and example filters - http://grokdebug.herokuapp.com/ - http://www.ruby-doc.org/stdlib-1.9.3/libdoc/date/rdoc/DateTime.html#method-i-strftime
Log2json lets you read, filter and send logs as JSON objects via Unix pipes. It is inspired by Logstash, and is meant to be compatible with it at the JSON event/record level so that it can easily work with Kibana. Reading logs is done via a shell script(eg, `tail`) running in its own process. You then configure(see the `syslog2json` or the `nginxlog2json` script for examples) and run your filters in Ruby using the `Log2Json` module and its contained helper classes. `Log2Json` reads from stdin the logs(one log record per line), parses the log lines into JSON records, and then serializes and writes the records to stdout, which then can be piped to another process for processing or sending it to somewhere else. Currently, Log2json ships with a `tail-log` script that can be run as the input process. It's the same as using the Linux `tail` utility with the `-v -F` options except that it also tracks the positions(as the numbers of lines read from the beginning of the files) in a few files in the file system so that if the input process is interrupted, it can continue reading from where it left off next time if the files had been followed. This feature is similar to the sincedb feature in Logstash's file input. Note: If you don't need the tracking feature(ie, you are fine with always tailling from the end of file with `-v -F -n0`), then you can just use the `tail` utility that comes with your Linux distribution.(Or more specifically, the `tail` from the GNU coreutils). Other versions of the `tail` utility may also work, but are not tested. The input protocol expected by Log2json is very simple and documented in the source code. ** The `tail-log` script uses a patched version of `tail` from the GNU coreutils package. A binary of the `tail` utility compiled for Ubuntu 12.04 LTS is included with the Log2json gem. If the binary doesn't work for your distribution, then you'll need to get GNU coreutils-8.13, apply the patch(it can be found in the src/ directory of the installed gem), and then replace the bin/tail binary in the directory of the installed gem with your version of the binary. ** P.S. If you know of a way to configure and compile ONLY the tail program in coreutils, please let me know! The reason I'm not building tail post gem installation is that it takes too long to configure && make because that actually builds every utilties in coreutils. For shipping logs to Redis, there's the `lines2redis` script that can be used as the output process in the pipe. For shipping logs from Redis to ElasticSearch, Log2json provides a `redis2es` script. Finally here's an example of Log2json in action: From a client machine: tail-log /var/log/{sys,mail}log /var/log/{kern,auth}.log | syslog2json | queue=jsonlogs \ flush_size=20 \ flush_interval=30 \ lines2redis host.to.redis.server 6379 0 # use redis DB 0 On the Redis server: redis_queue=jsonlogs redis2es host.to.es.server Resources that help writing log2json filters: - look at log2json.rb source and example filters - http://grokdebug.herokuapp.com/ - http://www.ruby-doc.org/stdlib-1.9.3/libdoc/date/rdoc/DateTime.html#method-i-strftime
minitest provides a complete suite of testing facilities supporting TDD, BDD, and benchmarking. "I had a class with Jim Weirich on testing last week and we were allowed to choose our testing frameworks. Kirk Haines and I were paired up and we cracked open the code for a few test frameworks... I MUST say that minitest is *very* readable / understandable compared to the 'other two' options we looked at. Nicely done and thank you for helping us keep our mental sanity." -- Wayne E. Seguin minitest/test is a small and incredibly fast unit testing framework. It provides a rich set of assertions to make your tests clean and readable. minitest/spec is a functionally complete spec engine. It hooks onto minitest/test and seamlessly bridges test assertions over to spec expectations. minitest/benchmark is an awesome way to assert the performance of your algorithms in a repeatable manner. Now you can assert that your newb co-worker doesn't replace your linear algorithm with an exponential one! minitest/pride shows pride in testing and adds coloring to your test output. I guess it is an example of how to write IO pipes too. :P minitest/test is meant to have a clean implementation for language implementors that need a minimal set of methods to bootstrap a working test suite. For example, there is no magic involved for test-case discovery. "Again, I can't praise enough the idea of a testing/specing framework that I can actually read in full in one sitting!" -- Piotr Szotkowski Comparing to rspec: rspec is a testing DSL. minitest is ruby. -- Adam Hawkins, "Bow Before MiniTest" minitest doesn't reinvent anything that ruby already provides, like: classes, modules, inheritance, methods. This means you only have to learn ruby to use minitest and all of your regular OO practices like extract-method refactorings still apply. == Features/Problems: * minitest/autorun - the easy and explicit way to run all your tests. * minitest/test - a very fast, simple, and clean test system. * minitest/spec - a very fast, simple, and clean spec system. * minitest/benchmark - an awesome way to assert your algorithm's performance. * minitest/pride - show your pride in testing! * minitest/test_task - a full-featured and clean rake task generator. * Incredibly small and fast runner, but no bells and whistles. * Written by squishy human beings. Software can never be perfect. We will all eventually die.
No description provided.
No description provided.
No description provided.
No description provided.
No description provided.
No description provided.
No description provided.
No description provided.
No description provided.
No description provided.
No description provided.
No description provided.