Iterate over streams (actually iterators) by line.
An ESnext spec-compliant iterator helpers shim/polyfill/replacement that works as far down as ES3.
Convert a reader into a async iterator emitting text lines
Firefox 17-26 iterators throw a StopIteration object to indicate "done". This normalizes it.
Convert an argument into a valid iterator. Based on the `.makeIterator()` implementation in mout https://github.com/mout/mout.
Iterator abstraction based on ES6 specification
Get an iterator for any JS language value. Works robustly across all environments, all versions.
Higher order iterator library for JavaScript/TypeScript.
A finite state machine iterator for JavaScript
Iterate any JS iterator. Works robustly in all environments, all versions.
Map over array avoiding parallel execution, using promises
Iterate over promises serially
A polymorphic iterate operator for arrays and other iterables
Turn an abstract-leveldown iterator into a readable stream
[](http://www.typescriptlang.org/) [](https://www.npmjs.com/package/@n1ru4l/push-pull-async
Creates an async iterator for a variety of inputs in the browser and node. Supports fetch, node-fetch, and cross-fetch
Asynchronous iterator function for parallel processing.
Framework-independent loaders for 3D graphics formats
Run multiple promise-returning & async functions with limited concurrency using native ES9
A thing that is a lot like ES6 `Map`, but without iterators, for use in environments where `for..of` syntax and `Map` are not available.
Get the default iterator or async iterator for an iterable or async iterable
Iterate any iterable JS value. Works robustly in all environments, all versions.
Simple iterator for flat and multi section lists
async.mapLimit's functionality available as a standalone npm module
Provides methods to more easily work with line-oriented text file and records within those files
This gem provides a (hopefully) high quality http parser library that can build request information iteratively as data comes over the line without requiring the caller to maintain the entire body of the request as a single string in memory.
Sangaku Eyeball uses Gosu, a 2D Game Library, to display lines, polygons and so on as defined by the Sangaku gem. It allows you to draw arbitrary polygons and then locate the Pole of Inaccessibility (usually the best place to display a label) using a fast, iterative algorithm.
Mutante iterates through lines of code, comments each one out, runs the associated tests, and flags lines whose removal leaves the suite green — strong candidates for dead code.
Descriptive configuration files for Ruby written in Ruby. Loquacious provides a very open configuration system written in ruby and descriptions for each configuration attribute. The attributes and descriptions can be iterated over allowing for helpful information about those attributes to be displayed to the user. In the simple case we have a file something like Loquacious.configuration_for('app') { name 'value', :desc => "Defines the name" foo 'bar', :desc => "FooBar" id 42, :desc => "Ara T. Howard" } Which can be loaded via the standard Ruby loading mechanisms Kernel.load 'config/app.rb' The attributes and their descriptions can be printed by using a Help object help = Loquacious.help_for('app') help.show :values => true # show the values for the attributes, too Descriptions are optional, and configurations can be nested arbitrarily deep. Loquacious.configuration_for('nested') { desc "The outermost level" a { desc "One more level in" b { desc "Finally, a real value" c 'value' } } } config = Loquacious.configuration_for('nested') p config.a.b.c #=> "value" And as you can see, descriptions can either be given inline after the value or they can appear above the attribute and value on their own line.
Descriptive configuration files for Ruby written in Ruby. Loquacious provides a very open configuration system written in ruby and descriptions for each configuration attribute. The attributes and descriptions can be iterated over allowing for helpful information about those attributes to be displayed to the user. In the simple case we have a file something like: Loquacious.configuration_for('app') { name 'value', :desc => "Defines the name" foo 'bar', :desc => "FooBar" id 42, :desc => "Ara T. Howard" } Which can be loaded via the standard Ruby loading mechanisms load 'config/app.rb' The attributes and their descriptions can be printed by using a Help object help = Loquacious.help_for('app') help.show :values => true # show the values for the attributes, too Descriptions are optional, and configurations can be nested arbitrarily deep. Loquacious.configuration_for('nested') { desc "The outermost level" a { desc "One more level in" b { desc "Finally, a real value" c 'value' } } } config = Loquacious.configuration_for 'nested' p config.a.b.c #=> "value" And as you can see, descriptions can either be given inline after the value or they can appear above the attribute and value on their own line.
Descriptive configuration files for Ruby written in Ruby. Loquacious provides a very open configuration system written in ruby and descriptions for each configuration attribute. The attributes and descriptions can be iterated over allowing for helpful information about those attributes to be displayed to the user. In the simple case we have a file something like Loquacious.configuration_for('app') { name 'value', :desc => "Defines the name" foo 'bar', :desc => "FooBar" id 42, :desc => "Ara T. Howard" } Which can be loaded via the standard Ruby loading mechanisms Kernel.load 'config/app.rb' The attributes and their descriptions can be printed by using a Help object help = Loquacious.help_for('app') help.show :values => true # show the values for the attributes, too Descriptions are optional, and configurations can be nested arbitrarily deep. Loquacious.configuration_for('nested') { desc "The outermost level" a { desc "One more level in" b { desc "Finally, a real value" c 'value' } } } config = Loquacious.configuration_for('nested') p config.a.b.c #=> "value" And as you can see, descriptions can either be given inline after the value or they can appear above the attribute and value on their own line.
Crowdfund is a Ruby program developed based on Pragmatic Studio's Ruby Programming hands-on video course, and distributed as a Ruby gem. This program has been developed using all the strengths of Ruby including the following. Ruby Programming Environment * Installing Ruby on your favorite operating system (free exercise) * Running Ruby using the interactive Ruby shell (irb) and writing Ruby program files * Using Ruby's documentation system to get help * Installing external Ruby libraries using RubyGems * Troubleshooting common problems Ruby Language Constructs * Expressions and variables * Numbers, string, and symbols (free video & exercise) * Loops and conditional expressions * Arrays and hashes (free video & exercise on hashes) * Classes, modules, and structs Object-Oriented Programming * Using built-in Ruby classes * Defining your own classes with state and behavior (free video & exercise) * Creating unique objects * Telling objects what to do by calling methods * Modeling class-level inheritance relationships * Sharing code with mixins Object-Oriented Design Principles * Encapsulation * Separation of concerns * Polymorphism * Don't Repeat Yourself * Tell, Don't Ask Blocks and Iterators * Calling built-in methods that take blocks * Writing your own methods that yield to blocks * Implementing custom iterators * Effectively using blocks in your programs Organizing Ruby Code * Creating a Ruby project structure * Separating source files for easier reuse and testing * Namespacing to avoid naming clashes * Input/Output * Reading data from files * Writing data to files * Creating an interactive console prompt * Handling command-line input Unit Testing * Writing and running unit tests with RSpec * Test-driven development and the red-green-refactor cycle * Stubbing methods to control tests * Refactoring code, safely! Distribution * Conforming to RubyGems conventions * Writing a GemSpec * Building a RubyGem * Publishing a RubyGem to a public server Ruby Programming Idioms
Studio Game is a Ruby program developed based on Pragmatic Studio' Ruby Programming hands-on video course, and distributed as a Ruby gem. This program has been developed using all the strengths of Ruby including the following. Ruby Programming Environment * Installing Ruby on your favorite operating system (free exercise) * Running Ruby using the interactive Ruby shell (irb) and writing Ruby program files * Using Ruby's documentation system to get help * Installing external Ruby libraries using RubyGems * Troubleshooting common problems Ruby Language Constructs * Expressions and variables * Numbers, string, and symbols (free video & exercise) * Loops and conditional expressions * Arrays and hashes (free video & exercise on hashes) * Classes, modules, and structs Object-Oriented Programming * Using built-in Ruby classes * Defining your own classes with state and behavior (free video & exercise) * Creating unique objects * Telling objects what to do by calling methods * Modeling class-level inheritance relationships * Sharing code with mixins Object-Oriented Design Principles * Encapsulation * Separation of concerns * Polymorphism * Don't Repeat Yourself * Tell, Don't Ask Blocks and Iterators * Calling built-in methods that take blocks * Writing your own methods that yield to blocks * Implementing custom iterators * Effectively using blocks in your programs Organizing Ruby Code * Creating a Ruby project structure * Separating source files for easier reuse and testing * Namespacing to avoid naming clashes * Input/Output * Reading data from files * Writing data to files * Creating an interactive console prompt * Handling command-line input Unit Testing * Writing and running unit tests with RSpec * Test-driven development and the red-green-refactor cycle * Stubbing methods to control tests * Refactoring code, safely! Distribution * Conforming to RubyGems conventions * Writing a GemSpec * Building a RubyGem * Publishing a RubyGem to a public server Ruby Programming Idioms
Send SMS messages using the CellForce API
== ICU4R - ICU Unicode bindings for Ruby ICU4R is an attempt to provide better Unicode support for Ruby, where it lacks for a long time. Current code is mostly rewritten string.c from Ruby 1.8.3. ICU4R is Ruby C-extension binding for ICU library[1] and provides following classes and functionality: * UString: - String-like class with internal UTF16 storage; - UCA rules for UString comparisons (<=>, casecmp); - encoding(codepage) conversion; \ - Unicode normalization; - transliteration, also rule-based; Bunch of locale-sensitive functions: - upcase/downcase; - string collation; \ - string search; - iterators over text line/word/char/sentence breaks; \ - message formatting (number/currency/string/time); - date and number parsing. * URegexp - unicode regular expressions. * UResourceBundle - access to resource bundles, including ICU locale data. * UCalendar - date manipulation and timezone info. * UConverter - codepage conversions API * UCollator - locale-sensitive string comparison == Install and usage > ruby extconf.rb > make && make check > make install Now, in your scripts just require 'icu4r'. To create RDoc, run > sh tools/doc.sh == Requirements To build and use ICU4R you will need GCC and ICU v3.4 libraries[2]. == Differences from Ruby String and Regexp classes === UString vs String 1. UString substring/index methods use UTF16 codeunit indexes, not code points. 2. UString supports most methods from String class. Missing methods are: capitalize, capitalize!, swapcase, swapcase! %, center, ljust, rjust chomp, chomp!, chop, chop! \ count, delete, delete!, squeeze, squeeze!, tr, tr!, tr_s, tr_s! crypt, intern, sum, unpack dump, each_byte, each_line hex, oct, to_i, to_sym reverse, reverse! succ, succ!, next, next!, upto 3. Instead of String#% method, UString#format is provided. See FORMATTING for short reference. 4. UStrings can be created via String.to_u(encoding='utf8') or global u(str,[encoding='utf8']) calls. Note that +encoding+ parameter must be value of String class. 5. There's difference between character grapheme, codepoint and codeunit. See UNICODE reports for gory details, but in short: locale dependent notion of character can be presented using more than one codepoint - base letter and combining (accents) (also possible more than one!), and each codepoint can require more than one codeunit to store (for UTF8 codeunit size is 8bit, though \ some codepoints require up to 4bytes). So, UString has normalization and locale dependent break iterators. 6. Currently UString doesn't include Enumerable module. 7. UString index/[] methods which accept URegexp, throw exception if Regexp passed. 8. UString#<=>, UString#casecmp use UCA rules. === URegexp UString uses ICU regexp library. Pattern syntax is described in [./docs/UNICODE_REGEXPS] and ICU docs. There are some differences between processing in Ruby Regexp and URegexp: 1. When UString#sub, UString#gsub are called with block, special vars ($~, $&, $1, ...) aren't set, as their values are processed through deep ruby core code. Instead, block receives UMatch object, which is essentially immutable array of matching groups: "test".u.gsub(ure("(e)(.)")) do |match| \ puts match[0] # => 'es' <--> $& puts match[1] # => 'e' \ <--> $1 puts match[2] # => 's' <--> $2 end 2. In URegexp search pattern backreferences are in form \n (\1, \2, ...), in replacement string - in form $1, $2, ... NOTE: URegexp considers char to be a digit NOT ONLY ASCII (0x0030-0x0039), but any Unicode char, which has property Decimal digit number (Nd), e.g.: a = [?$, 0x1D7D9].pack("U*").u * 2 puts a.inspect_names <U000024>DOLLAR SIGN <U01D7D9>MATHEMATICAL DOUBLE-STRUCK DIGIT ONE <U000024>DOLLAR SIGN <U01D7D9>MATHEMATICAL DOUBLE-STRUCK DIGIT ONE puts "abracadabra".u.gsub(/(b)/.U, a) abbracadabbra \ 3. One can create URegexp using global Kernel#ure function, Regexp#U, Regexp#to_u, or from UString using URegexp.new, e.g: /pattern/.U =~ "string".u 4. There are differences about Regexp and URegexp multiline matching options: t = "text\ntest" # ^,$ handling : URegexp multiline <-> Ruby default t.u =~ ure('^\w+$', URegexp::MULTILINE) => #<UMatch:0xf6f7de04 @ranges=[0..3], @cg=[\u0074\u0065\u0078\u0074]> t =~ /^\w+$/ => 0 # . matches \n : URegexp DOTALL <-> /m t.u =~ ure('.+test', URegexp::DOTALL) \ => #<UMatch:0xf6fa4d88 ... t.u =~ /.+test/m 5. UMatch.range(idx) returns range for capturing group idx. This range is in codeunits. === References 1. ICU Official Homepage http://ibm.com/software/globalization/icu/ 2. ICU downloads \ http://ibm.com/software/globalization/icu/downloads.jsp 3. ICU Home Page http://icu.sf.net 4. Unicode Home Page http://www.unicode.org ==== BUGS, DOCS, TO DO The code is slow and inefficient yet, is still highly experimental, so can have many security and memory leaks, bugs, inconsistent documentation, incomplete test suite. Use it at your own risk. Bug reports and feature requests are welcome :) === Copying This extension module is copyrighted free software by Nikolai Lugovoi. You can redistribute it and/or modify it under the terms of MIT License. Nikolai Lugovoi <meadow.nnick@gmail.com>
README ====== This is a simple API to evaluate information retrieval results. It allows you to load ranked and unranked query results and calculate various evaluation metrics (precision, recall, MAP, kappa) against a previously loaded gold standard. Start this program from the command line with: retreval -l <gold-standard-file> -q <query-results> -f <format> -o <output-prefix> The options are outlined when you pass no arguments and just call retreval You will find further information in the RDOC documentation and the HOWTO section below. If you want to see an example, use this command: retreval -l example/gold_standard.yml -q example/query_results.yml -f yaml -v INSTALLATION ============ If you have RubyGems, just run gem install retreval You can manually download the sources and build the Gem from there by `cd`ing to the folder where this README is saved and calling gem build retreval.gemspec This will create a gem file called which you just have to install with `gem install <file>` and you're done. HOWTO ===== This API supports the following evaluation tasks: - Loading a Gold Standard that takes a set of documents, queries and corresponding judgements of relevancy (i.e. "Is this document relevant for this query?") - Calculation of the _kappa measure_ for the given gold standard - Loading ranked or unranked query results for a certain query - Calculation of _precision_ and _recall_ for each result - Calculation of the _F-measure_ for weighing precision and recall - Calculation of _mean average precision_ for multiple query results - Calculation of the _11-point precision_ and _average precision_ for ranked query results - Printing of summary tables and results Typically, you will want to use this Gem either standalone or within another application's context. Standalone Usage ================ Call parameters --------------- After installing the Gem (see INSTALLATION), you can always call `retreval` from the commandline. The typical call is: retreval -l <gold-standard-file> -q <query-results> -f <format> -o <output-prefix> Where you have to define the following options: - `gold-standard-file` is a file in a specified format that includes all the judgements - `query-results` is a file in a specified format that includes all the query results in a single file - `format` is the format that the files will use (either "yaml" or "plain") - `output-prefix` is the prefix of output files that will be created Formats ------- Right now, we focus on the formats you can use to load data into the API. Currently, we support YAML files that must adhere to a special syntax. So, in order to load a gold standard, we need a file in the following format: * "query" denotes the query * "documents" these are the documents judged for this query * "id" the ID of the document (e.g. its filename, etc.) * "judgements" an array of judgements, each one with: * "relevant" a boolean value of the judgment (relevant or not) * "user" an optional identifier of the user Example file, with one query, two documents, and one judgement: - query: 12th air force germany 1957 documents: - id: g5701s.ict21311 judgements: [] - id: g5701s.ict21313 judgements: - relevant: false user: 2 So, when calling the program, specify the format as `yaml`. For the query results, a similar format is used. Note that it is necessary to specify whether the result sets are ranked or not, as this will heavily influence the calculations. You can specify the score for a document. By "score" we mean the score that your retrieval algorithm has given the document. But this is not necessary. The documents will always be ranked in the order of their appearance, regardless of their score. Thus in the following example, the document with "07" at the end is the first and "25" is the last, regardless of the score. --- query: 12th air force germany 1957 ranked: true documents: - score: 0.44034874 document: g5701s.ict21307 - score: 0.44034874 document: g5701s.ict21309 - score: 0.44034874 document: g5701s.ict21311 - score: 0.44034874 document: g5701s.ict21313 - score: 0.44034874 document: g5701s.ict21315 - score: 0.44034874 document: g5701s.ict21317 - score: 0.44034874 document: g5701s.ict21319 - score: 0.44034874 document: g5701s.ict21321 - score: 0.44034874 document: g5701s.ict21323 - score: 0.44034874 document: g5701s.ict21325 --- query: 1612 ranked: true documents: - score: 1.0174774 document: g3290.np000144 - score: 0.763108 document: g3201b.ct000726 - score: 0.763108 document: g3400.ct000886 - score: 0.6359234 document: g3201s.ct000130 --- **Note**: You can also use the `plain` format, which will load the gold standard in a different way (but not the results): my_query my_document_1 false my_query my_document_2 true See that every query/document/relevancy pair is separated by a tabulator? You can also add the user's ID in the fourth column if necessary. Running the evaluation ----------------------- After you have specified the input files and the format, you can run the program. If needed, the `-v` switch will turn on verbose messages, such as information on how many judgements, documents and users there are, but this shouldn't be necessary. The program will first load the gold standard and then calculate the statistics for each result set. The output files are automatically created and contain a YAML representation of the results. Calculations may take a while depending on the amount of judgements and documents. If there are a thousand judgements, always consider a few seconds for each result set. Interpreting the output files ------------------------------ Two output files will be created: - `output_avg_precision.yml` - `output_statistics.yml` The first lists the average precision for each query in the query result file. The second file lists all supported statistics for each query in the query results file. For example, for a ranked evaluation, the first two entries of such a query result statistic look like this: --- 12th air force germany 1957: - :precision: 0.0 :recall: 0.0 :false_negatives: 1 :false_positives: 1 :true_negatives: 2516 :true_positives: 0 :document: g5701s.ict21313 :relevant: false - :precision: 0.0 :recall: 0.0 :false_negatives: 1 :false_positives: 2 :true_negatives: 2515 :true_positives: 0 :document: g5701s.ict21317 :relevant: false You can see the precision and recall for that specific point and also the number of documents for the contingency table (true/false positives/negatives). Also, the document identifier is given. API Usage ========= Using this API in another ruby application is probably the more common use case. All you have to do is include the Gem in your Ruby or Ruby on Rails application. For details about available methods, please refer to the API documentation generated by RDoc. **Important**: For this implementation, we use the document ID, the query and the user ID as the primary keys for matching objects. This means that your documents and queries are identified by a string and thus the strings should be sanitized first. Loading the Gold Standard ------------------------- Once you have loaded the Gem, you will probably start by creating a new gold standard. gold_standard = GoldStandard.new Then, you can load judgements into this standard, either from a file, or manually: gold_standard.load_from_yaml_file "my-file.yml" gold_standard.add_judgement :document => doc_id, :query => query_string, :relevant => boolean, :user => John There is a nice shortcut for the `add_judgement` method. Both lines are essentially the same: gold_standard.add_judgement :document => doc_id, :query => query_string, :relevant => boolean, :user => John gold_standard << :document => doc_id, :query => query_string, :relevant => boolean, :user => John Note the usage of typical Rails hashes for better readability (also, this Gem was developed to be used in a Rails webapp). Now that you have loaded the gold standard, you can do things like: gold_standard.contains_judgement? :document => "a document", :query => "the query" gold_standard.relevant? :document => "a document", :query => "the query" Loading the Query Results ------------------------- Now we want to create a new `QueryResultSet`. A query result set can contain more than one result, which is what we normally want. It is important that you specify the gold standard it belongs to. query_result_set = QueryResultSet.new :gold_standard => gold_standard Just like the Gold Standard, you can read a query result set from a file: query_result_set.load_from_yaml_file "my-results-file.yml" Alternatively, you can load the query results one by one. To do this, you have to create the results (either ranked or unranked) and then add documents: my_result = RankedQueryResult.new :query => "the query" my_result.add_document :document => "test_document 1", :score => 13 my_result.add_document :document => "test_document 2", :score => 11 my_result.add_document :document => "test_document 3", :score => 3 This result would be ranked, obviously, and contain three documents. Documents can have a score, but this is optional. You can also create an Array of documents first and add them altogether: documents = Array.new documents << ResultDocument.new :id => "test_document 1", :score => 20 documents << ResultDocument.new :id => "test_document 2", :score => 21 my_result = RankedQueryResult.new :query => "the query", :documents => documents The same applies to `UnrankedQueryResult`s, obviously. The order of ranked documents is the same as the order in which they were added to the result. The `QueryResultSet` will now contain all the results. They are stored in an array called `query_results`, which you can access. So, to iterate over each result, you might want to use the following code: query_result_set.query_results.each_with_index do |result, index| # ... end Or, more simply: for result in query_result_set.query_results # ... end Calculating statistics ---------------------- Now to the interesting part: Calculating statistics. As mentioned before, there is a conceptual difference between ranked and unranked results. Unranked results are much easier to calculate and thus take less CPU time. No matter if unranked or ranked, you can get the most important statistics by just calling the `statistics` method. statistics = my_result.statistics In the simple case of an unranked result, you will receive a hash with the following information: * `precision` - the precision of the results * `recall` - the recall of the results * `false_negatives` - number of not retrieved but relevant items * `false_positives` - number of retrieved but nonrelevant * `true_negatives` - number of not retrieved and nonrelevantv items * `true_positives` - number of retrieved and relevant items In case of a ranked result, you will receive an Array that consists of _n_ such Hashes, depending on the number of documents. Each Hash will give you the information at a certain rank, e.g. the following to lines return the recall at the fourth rank. statistics = my_ranked_result.statistics statistics[3][:recall] In addition to the information mentioned above, you can also get for each rank: * `document` - the ID of the document that was returned at this rank * `relevant` - whether the document was relevant or not Calculating statistics with missing judgements ---------------------------------------------- Sometimes, you don't have judgements for all document/query pairs in the gold standard. If this happens, the results will be cleaned up first. This means that every document in the results that doesn't appear to have a judgement will be removed temporarily. As an example, take the following results: * A * B * C * D Our gold standard only contains judgements for A and C. The results will be cleaned up first, thus leading to: * A * C With this approach, we can still provide meaningful results (for precision and recall). Other statistics ---------------- There are several other statistics that can be calculated, for example the **F measure**. The F measure weighs precision and recall and has one parameter, either "alpha" or "beta". Get the F measure like so: my_result.f_measure :beta => 1 If you don't specify either alpha or beta, we will assume that beta = 1. Another interesting measure is **Cohen's Kappa**, which tells us about the inter-agreement of assessors. Get the kappa statistic like this: gold_standard.kappa This will calculate the average kappa for each pairwise combination of users in the gold standard. For ranked results one might also want to calculate an **11-point precision**. Just call the following: my_ranked_result.eleven_point_precision This will return a Hash that has indices at the 11 recall levels from 0 to 1 (with steps of 0.1) and the corresponding precision at that recall level.