A writeFile queue which reattempts to write after errors occur
Write files in an atomic fashion w/configurable ownership
Tiny queue data structure
A shim for the setImmediate efficient script yielding API
Specialized fast async file writer
fast, tiny `queueMicrotask` shim for modern engines
Offload tasks to a pool of workers on node.js and in the browser
simplified stream construction
Read and write files atomically and reliably.
Simple JS queue with auto run for node and browsers
Promise queue with concurrency control
The smallest and simplest JavaScript priority queue
A simple tool to keep requests to be executed in order.
Simple JS queue with auto run for node and browsers
fs read and write streams based on minipass
queue-lit is a tiny queue data structure in case you `Array#push()` or `Array#shift()` on large arrays very often
Next tick shim that prefers process.nextTick over queueMicrotask for compat
OCI NodeJS client for Queue Service
fs read and write streams based on minipass
Read and parse a JSON file
simplified stream construction
The Node.js bindings of the humanfs library.
In memory queue system prioritizing tasks
Promise-based queue
Unified logger that writes to a log file, redis queue, mongo collection, and hoptoad.
Log2json lets you read, filter and send logs as JSON objects via Unix pipes. It is inspired by Logstash, and is meant to be compatible with it at the JSON event/record level so that it can easily work with Kibana. Reading logs is done via a shell script(eg, `tail`) running in its own process. You then configure(see the `syslog2json` or the `nginxlog2json` script for examples) and run your filters in Ruby using the `Log2Json` module and its contained helper classes. `Log2Json` reads from stdin the logs(one log record per line), parses the log lines into JSON records, and then serializes and writes the records to stdout, which then can be piped to another process for processing or sending it to somewhere else. Currently, Log2json ships with a `tail-log` script that can be run as the input process. It's the same as using the Linux `tail` utility with the `-v -F` options except that it also tracks the positions(as the numbers of lines read from the beginning of the files) in a few files in the file system so that if the input process is interrupted, it can continue reading from where it left off next time if the files had been followed. This feature is similar to the sincedb feature in Logstash's file input. Note: If you don't need the tracking feature(ie, you are fine with always tailling from the end of file with `-v -F -n0`), then you can just use the `tail` utility that comes with your Linux distribution.(Or more specifically, the `tail` from the GNU coreutils). Other versions of the `tail` utility may also work, but are not tested. The input protocol expected by Log2json is very simple and documented in the source code. ** The `tail-log` script uses a patched version of `tail` from the GNU coreutils package. A binary of the `tail` utility compiled for Ubuntu 12.04 LTS is included with the Log2json gem. If the binary doesn't work for your distribution, then you'll need to get GNU coreutils-8.13, apply the patch(it can be found in the src/ directory of the installed gem), and then replace the bin/tail binary in the directory of the installed gem with your version of the binary. ** P.S. If you know of a way to configure and compile ONLY the tail program in coreutils, please let me know! The reason I'm not building tail post gem installation is that it takes too long to configure && make because that actually builds every utilties in coreutils. For shipping logs to Redis, there's the `lines2redis` script that can be used as the output process in the pipe. For shipping logs from Redis to ElasticSearch, Log2json provides a `redis2es` script. Finally here's an example of Log2json in action: From a client machine: tail-log /var/log/{sys,mail}log /var/log/{kern,auth}.log | syslog2json | queue=jsonlogs \ flush_size=20 \ flush_interval=30 \ lines2redis host.to.redis.server 6379 0 # use redis DB 0 On the Redis server: redis_queue=jsonlogs redis2es host.to.es.server Resources that help writing log2json filters: - look at log2json.rb source and example filters - http://grokdebug.herokuapp.com/ - http://www.ruby-doc.org/stdlib-1.9.3/libdoc/date/rdoc/DateTime.html#method-i-strftime
= DESCRIPTION: Provides a Chef handler which can report run status, including any changes that were made, to a rabbit server. In the case of failed runs a backtrace will be included in the details reported. Based on the Graylog Gelf handler by Jon Wood (<jon@blankpad.net>) https://github.com/jellybob/chef-gelf = REQUIREMENTS: * A Rabbit server running somewhere. = USAGE: This example makes of the chef_handler cookbook, place some thing like this in cookbooks/chef_handler/recipes/rabbit.rb and add it to your run list. include_recipe "chef_handler::default" gem_package "chef-rabbit" do action :nothing end.run_action(:install) # Make sure the newly installed Gem is loaded. Gem.clear_paths require 'chef/rabbit' chef_handler "Chef::RABBIT::Handler" do source "chef/rabbit" arguments({ :connection => { :host => "your_rabbit_server", :user => "rabbit_user", :pass => "rabbit_pass", :vhost => "/stuff" } :queue => { :name => "some_queue", :params => { :durable => true, ... } }, :exchange => { :name => "some_exchange", :params => { :durable => true, ... } }, :timestamp_tag => "@timestamp" }) supports :exception => true, :report => true end.run_action(:enable) Arguments take the form of an options hash, with the following options: * :connection - http://rubybunny.info/articles/connecting.html * :queue - rabbit queue info to use. name is set to "chef-client" + durable = true by default * :exchange - rabbit exchange to use .default_exchange + durable = true by default * :timestamp_tag - tag for timestamp "timestamp" by default * :blacklist ({}) - A hash of cookbooks, resources and actions to ignore in the change list. = BLACKLISTING: Some resources report themselves as having updated on every run even if nothing changed, or are just things you don't care about. To reduce the amount of noise in your logs these can be ignored by providing a blacklist. In this example we don't want to be told about the GELF handler being activated: chef_handler "Chef::RABBIT::Handler" do source "chef/rabbit" arguments({ :blacklist => { "chef_handler" => { "chef_handler" => [ "nothing", "enable" ] } } }) supports :exception => true, :report => true end.run_action(:enable) = LICENSE and AUTHOR: Copyright 2014 by MTN Satellite Communications Licensed under the Apache License, Version 2.0 (the “License”); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an “AS IS” BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
Log2json lets you read, filter and send logs as JSON objects via Unix pipes. It is inspired by Logstash, and is meant to be compatible with it at the JSON event/record level so that it can easily work with Kibana. Reading logs is done via a shell script(eg, `tail`) running in its own process. You then configure(see the `syslog2json` or the `nginxlog2json` script for examples) and run your filters in Ruby using the `Log2Json` module and its contained helper classes. `Log2Json` reads from stdin the logs(one log record per line), parses the log lines into JSON records, and then serializes and writes the records to stdout, which then can be piped to another process for processing or sending it to somewhere else. Currently, Log2json ships with a `tail-log` script that can be run as the input process. It's the same as using the Linux `tail` utility with the `-v -F` options except that it also tracks the positions(as the numbers of lines read from the beginning of the files) in a few files in the file system so that if the input process is interrupted, it can continue reading from where it left off next time if the files had been followed. This feature is similar to the sincedb feature in Logstash's file input. Note: If you don't need the tracking feature(ie, you are fine with always tailling from the end of file with `-v -F -n0`), then you can just use the `tail` utility that comes with your Linux distribution.(Or more specifically, the `tail` from the GNU coreutils). Other versions of the `tail` utility may also work, but are not tested. The input protocol expected by Log2json is very simple and documented in the source code. ** The `tail-log` script uses a patched version of `tail` from the GNU coreutils package. A binary of the `tail` utility compiled for Ubuntu 12.04 LTS is included with the Log2json gem. If the binary doesn't work for your distribution, then you'll need to get GNU coreutils-8.13, apply the patch(it can be found in the src/ directory of the installed gem), and then replace the bin/tail binary in the directory of the installed gem with your version of the binary. ** P.S. If you know of a way to configure and compile ONLY the tail program in coreutils, please let me know! The reason I'm not building tail post gem installation is that it takes too long to configure && make because that actually builds every utilties in coreutils. For shipping logs to Redis, there's the `lines2redis` script that can be used as the output process in the pipe. For shipping logs from Redis to ElasticSearch, Log2json provides a `redis2es` script. Finally here's an example of Log2json in action: From a client machine: tail-log /var/log/{sys,mail}log /var/log/{kern,auth}.log | syslog2json | queue=jsonlogs \ flush_size=20 \ flush_interval=30 \ lines2redis host.to.redis.server 6379 0 # use redis DB 0 On the Redis server: redis_queue=jsonlogs redis2es host.to.es.server Resources that help writing log2json filters: - look at log2json.rb source and example filters - http://grokdebug.herokuapp.com/ - http://www.ruby-doc.org/stdlib-1.9.3/libdoc/date/rdoc/DateTime.html#method-i-strftime
No description provided.
No description provided.
No description provided.