Results for better-context

AI Agent Codebase Intelligence CLI - Generate AGENTS.md hierarchies

Clone & update reference repos for AI coding agents.

Working Set is a multithreaded custom protocol server with an ncurses-based terminal interface that displays search results with context for better comprehension and easy navigation.

LLM Memory is a Ruby gem designed to provide large language models (LLMs) like ChatGPT with memory using in-context learning. This enables better integration with systems such as Rails and web services while providing a more user-friendly and abstract interface based on brain terms.

The pathname2 library provides an implementation of the Pathname class different from the one that ships as part of the Ruby standard library. It is a subclass of String, though several methods have been overridden to better fit a path context. In addition, it supports file URL's as paths, provides additional methods for Windows paths, and handles UNC paths on Windows properly. See the README file for more details.

Commit AI automates the generation of insightful single-line and multi-line commit messages by analyzing staged git diffs with OpenAI's advanced language models. It allows users to provide optional change descriptions for better context, and smartly minifies the git diff to focus on the essential changes, ensuring clarity and best practices.

Acronym Expander is an AI-powered tool for expanding acronyms to their full meanings. It uses a dictionary of common acronyms and AI for unknown ones, with optional context for better accuracy.

:title: The Ruby API :section: PYAPNS::Client There's python in my ruby! This is a class used to send notifications, provision applications and retrieve feedback using the Apple Push Notification Service. PYAPNS is a multi-application APS provider, meaning it is possible to send notifications to any number of different applications from the same application and same server. It is also possible to scale the client to any number of processes and servers, simply balanced behind a simple web proxy. It may seem like overkill for such a bare interface - after all, the APS service is rather simplistic. However, PYAPNS takes no shortcuts when it comes to completeness/compliance with the APNS protocol and allows the user many optimization and scaling vectors not possible with other libraries. No bandwidth is wasted, connections are persistent and the server is asynchronous therefore notifications are delivered immediately. PYAPNS takes after the design of 3rd party push notification service that charge a fee each time you push a notification, and charge extra for so-called 'premium' service which supposedly gives you quicker access to the APS servers. However, PYAPNS is free, as in beer and offers more scaling opportunities without the financial draw. :section: Provisioning To add your app to the PYAPNS server, it must be `provisioned` at least once. Normally this is done once upon the start-up of your application, be it a web service, desktop application or whatever... It must be done at least once to the server you're connecting to. Multiple instances of PYAPNS will have to have their applications provisioned individually. To provision an application manually use the `PYAPNS::Client#provision` method. require 'pyapns' client = PYAPNS::Client.configure client.provision :app_id => 'cf', :cert => '/home/ss/cert.pem', :env => 'sandbox', :timeout => 15 This basically says "add an app reference named 'cf' to the server and start a connection using the certification, and if it can't within 15 seconds, raise a `PYAPNS::TimeoutException` That's all it takes to get started. Of course, this can be done automatically by using PYAPNS::ClientConfiguration middleware. `PYAPNS::Client` is a singleton class that is configured using the class method `PYAPNS::Client#configure`. It is sensibly configured by default, but can be customized by specifying a hash See the docs on `PYAPNS::ClientConfiguration` for a list of available configuration parameters (some of these are important, and you can specify initial applications) to be configured by default. :section: Sending Notifications Once your client is configured, and application provisioned (again, these should be taken care of before you write notification code) you can begin sending notifications to users. If you're wondering how to acquire a notification token, you've come to the wrong place... I recommend using google. However, if you want to send hundreds of millions of notifications to users, here's how it's done, one at a time... The `PYAPNS::Client#notify` is a sort of polymorphic method which can notify any number of devices at a time. It's basic form is as follows: client.notify 'cf', 'long ass app token', {:aps=> {:alert => 'hello?'}} However, as stated before, it is sort of polymorphic: client.notify 'cf', ['token', 'token2', 'token3'], [alert, alert2, alert3] client.notify :app_id => 'cf', :tokens => 'mah token', :notifications => alertHash client.notify 'cf', 'token', PYAPNS::Notification('hello tits!') As you can see, the method accepts paralell arrays of tokens and notifications meaning any number of notifications can be sent at once. Hashes will be automatically converted to `PYAPNS::Notification` objects so they can be optimized for the wire (nil values removed, etc...), and you can pass `PYAPNS::Notification` objects directly if you wish. :section: Retrieving Feedback The APS service offers a feedback functionality that allows application servers to retrieve a list of device tokens it deems to be no longer in use, and the time it thinks they stopped being useful (the user uninstalled your app, better luck next time...) Sounds pretty straight forward, and it is. Apple recommends you do this at least once an hour. PYAPNS will return a list of 2-element lists with the date and the token: feedbacks = client.feedback 'cf' :section: Asynchronous Calls PYAPNS::Client will, by default, perform no funny stuff and operate entirely within the calling thread. This means that certain applications may hang when, say, sending a notification, if only for a fraction of a second. Obviously not a desirable trait, all `provision`, `feedback` and `notify` methods also take a block, which indicates to the method you want to call PYAPNS asynchronously, and it will be done so handily in another thread, calling back your block with a single argument when finished. Note that `notify` and `provision` return absolutely nothing (nil, for you rub--wait you are ruby developers!). It is probably wise to always use this form of operation so your calling thread is never blocked (especially important in UI-driven apps and asynchronous servers) Just pass a block to provision/notify/feedback like so: PYAPNS::Client.instance.feedback do |feedbacks| feedbacks.each { |f| trim_token f } end :section: PYAPNS::ClientConfiguration A middleware class to make `PYAPNS::Client` easy to use in web contexts Automates configuration of the client in Rack environments using a simple confiuration middleware. To use `PYAPNS::Client` in Rack environments with the least code possible `use PYAPNS::ClientConfiguration` (no, really, in some cases, that's all you need!) middleware with an optional hash specifying the client variables. Options are as follows: use PYAPNS::ClientConfiguration( :host => 'http://localhost/' :port => 7077, :initial => [{ :app_id => 'myapp', :cert => '/home/myuser/apps/myapp/cert.pem', :env => 'sandbox', :timeout => 15 }]) Where the configuration variables are defined: :host String the host where the server can be found :port Number the port to which the client should connect :initial Array OPTIONAL - an array of INITIAL hashes INITIAL HASHES: :app_id String the id used to send messages with this certification can be a totally arbitrary value :cert String a path to the certification or the certification file as a string :env String the environment to connect to apple with, always either 'sandbox' or 'production' :timoeut Number The timeout for the server to use when connecting to the apple servers :section: PYAPNS::Notification An APNS Notification You can construct notification objects ahead of time by using this class. However unnecessary, it allows you to programmatically generate a Notification like so: note = PYAPNS::Notification.new 'alert text', 9, 'flynn.caf', {:extra => 'guid'} -- or -- note = PYAPNS::Notification.new 'alert text' These can be passed to `PYAPNS::Client#notify` the same as hashes

# foundationallib <h2>Finally, a cross-platform, portable, well-designed, secure, robust, maximally-efficient C foundational library &mdash; Making Engineering And Computing Fast, Secure, Responsive And Easy.</h2> <br> <h2><i>Library Uses - What It Does, What It Is, And What It Is A Solution For</i></h2> <ul class="features-list"> <li><strong>Enables better Engineering Solutions and Security broadly and foundationally where Software Creation or Development or Script Creation is concerned - whether this be on a local, business, governmental or international basis, and makes things easier - and Computing in General.</strong> Don't Reinvent the Wheel - Use Good Wheels - Be Safe And Secure.</li> <br> <li><strong>Enables a free-flowing dynamic computer usage that you need, deserve and should have, simply because you have a computer. With full speed and with robustness. You deserve to be able to use your computer wholly and fully, with proper and fast operations.</strong></li> <br><li><strong>Enables flexibility and power - makes C accessible to the masses (and faster and more secure) with easy usage and strives to bring people up, not degrade the character or actions of people.</strong> This is a fundamental and unequivocal philosophy difference between this library and many subsections of Software Engineering and the mainstream engineering establishment. For instance, in Python, you cannot read a file easily &ndash; you have to read it line-by-line or open a file, read the lines, then close it. With this library, you can efficiently read 10,000 files in one function call. This library gives power. Any common operation, there ought to be a powerful function for.<br><br>We should not bitch around with assembly when we don't want to; we should also have full speed. Some old "solutions" deliver neither, then culturally degrade programmers because their tools are bad - actually, it just degrades programmers, and gives them bad tools. COBOL is an example ...<br><br>Human technology is about empowerment &ndash; people must fight for it to be empowerment, we don't have time to have AI systems kill us because we want to have bad tools and be weak. We must fight.</li> </ul> <br> <ul> <h2><i>About Foundationallib</i></h2> <li>→<strong>Cross platform</strong> - works perfectly in embedded, server, desktop, and all platforms - tested for Windows and UNIX - 64-bit and 32-bit, includes a 3-aspect test suite, with more to come.</li> <li>→<strong>Bug free. Reliable. Dependable. Secure. Tested well.</strong></li> <li>→<strong>Zero Overhead</strong> - Only 1 byte due to the power of the error handling, can be configured will full power.</li> <li>→<strong>Static Inline Functions if you want them</strong> (optional) - Eliminating function call overhead to 0 if you wish, for improved performance.</li> <li>→<strong>Custom allocators</strong> - if you want it.</li> <li>→<strong>Custom error handling</strong> - if you want it.</li> <li>→<strong>Safe functions</strong> warn the programmer about NULL values and unused return values. Can be configured to not compile if not Secure. Optional null-check macros in every library function. Does not use any of <code>"gets", "fgets", "strcpy", "strcat", "sprintf", "vsprintf", "scanf", "fscanf", "system", "chown", "chmod", "chgrp", "alloca", "execl", "execle", "execlp", "execv", "execve", "execvp", "bcopy", "bzero"</code>. You can configure it to never use any unsafe functions.</li> <li>→<strong>Portable</strong> - works on all platforms, using platform specific features (using #ifdefs) to make functions better and faster.</li> <li>→<strong>Multithreading support</strong> (optional), with list_comprehension_multithreaded (accepts any number of threads, works in parallel using portable C11 threads)</li> <li>→<strong>Networking support</strong> (optional), using libcurl - making it extremely easy to download websites and arrays of websites - features other languages do not have.</li> <li>→Very good and thorough <strong>Error Handling</strong> and <strong>allocation overflow</strong> checking (good for <strong>Security and Robustness</strong>) in the functions. Allows the programmer to dynamically choose to catch all errors in the functions with a handler (default or custom), or to ignore them. No need to ALWAYS say "if (.....) if you don't want to. Can be changed at runtime.</li> <li>→<strong>Public Domain</strong> so you make the code how you want. (No need to "propitiate" to some "god" of some library).</li> <li>→<strong>Minimal abstractions or indirection of any kind or needless slow things that complicate things</strong> - macros, namespace collision, typedefs, structs, object-orientation messes, slow compilation times, bloat, etc., etc.</li> <li>→<strong>No namespace pollution</strong> - you can generate your <span style=font-style:normal;><b>own version</b></span> with any prefix you like!</li> <li>→<strong>Relies <span style=font-style:normal;>minimally</span> on C libraries - it can be fully decoupled from LIB C and can be statically linked.</strong></li> <li>→<span style=font-style:normal;><b>Very small</b></span> - 13K Lines of Code (including Doxygen comments and following of Best Practices)</li> <li>→<strong>No Linkage Issues or dependency hell</strong></li> <li>→<strong>Thorough and clear documentation</strong>, with examples of usage.</li> <li>→<strong>No licensing restrictions whatsoever - use it for your engineering project, your startup, your Fortune 500 company, your personal project, your throw-away script, your government.</strong></li> <li>→<strong>Makes C like Python or Perl or Ruby in many ways - or more easy</strong></li> <li>→<strong>Easy Straightforward Transpilation Support</strong> - to make current code, much faster - all without any bloat (See transpile_slow_scripting_into_c.rb). <li><h4>In many cases, there is now a direct mapping of functions from other languages into optimized C. See the example script in this repository. This makes optimizing your Python / Perl / Ruby / PHP etc. script very easy, either manually or through the use of AI.</h4></li> </ul> </p> </div> <div class=pane style='border: 0;border-right: 1px dotted rgb(200, 200, 200); background-color: rgb(255, 255, 190);'> <div class="library-details"><h2 style=color:green;><i>Foundationallib Features</i></h2> <p class=feature> <strong>Functional Programming Features</strong> - <code>map, reduce, filter,</code> List Comprehensions in C and much more!</p> <p class=feature><strong>Expands C's Primitives for easy manipulation of data types</strong> such as Arrays, Strings, <code>Dict</code>, <code>Set</code>, <code>FrozenDict</code>, <code>FrozenSet</code> - <strong>and enables easy manipulation, modification, alteration, comparison, sorting, counting, IO (printing) and duplication of these at a very comfortable level</strong> - something very, very rare in C or C++, <i>all without any overhead.</i></p> <p class=feature><strong>More comfortable IO</strong> - read and write entire files with ease, and convert complex types into strings or print them on the screen with ease. </p> <p class=feature><strong>A powerful general purpose Foundational Library</strong> - <i>which has anything and everything you need</i> - from <code>replace_all()</code> to <code>replace_memory()</code> to <code>find_last_of()</code> to to <code>list_comprehension()</code> to <code>shellescape()</code> to <code>read_file_into_string()</code> to <code>string_to_json()</code> to <code>string_to_uppercase()</code> to <code>to_title_case()</code> to <code>read_file_into_array()</code> to <code>read_files_into_array()</code> to <code>map()</code> to <code>reduce()</code> to <code>filter()</code> to <code>list_comprehension_multithreaded()</code> to <code>frozen_dict_new_instance()</code> to <code>backticks()</code> - everything you would want to make quick and optimally efficient C programs, this has it.</p> <div style='height: 1px; border: 0;border-bottom: 1px dashed rgb(200, 200, 200);'></div> <p class=performance><span>Helps to make programs hundreds of times faster than other languages with similar ease of creation.</span> <hr> <p class=feature><strong>Easily take advantage of CPU cores with list_comprehension_multithreaded()</strong>.<br><br>You can specify the number of threads, the transform and the filter functions, and this will transform your data - all in parallel. Don't have a multithreaded environment? Then disable it (set the flag).</p> <hr> <h3>You don't want to be reinventing the wheel and hoping that your memory allocation is secure enough - and then failing. <strong>Security Is Paramount.</strong></h3> <h3>You don't want to be waiting <span style='color:rgb(240, 0, 0);'>a day</span> for an operation to complete when it could take <span style='color:rgb(30, 30, 255);'>less than an hour</span>.</h3> <br><p>This library is founded on very strong and unequivocal goals and philosophy. In fact, I have written many articles about the foundation of this library and more relevantly the broader context. See the Articles folder - for some of the foundation of this library.</p> <br><p>This library is an ideal and a dream - not just a Software Library. As such, I would highly suggest that you support me in this mission. Even if it's different from the status quo. Are you a Rust or Zig fan? Then make a Rust or Zig version of this ideal. Let's go. Give me an email.</p> </div> </div> <br> No Copyright - Public Domain - 2023, Gregory Cohen <gregorycohennew@gmail.com> DONATION REQUEST: If this free software has helped you and you find it valuable, please consider making a donation to support the ongoing development and maintenance of this project. Your contribution helps ensure the availability of this library to the community and encourages further improvements. Donations can be made at: https://www.paypal.com/paypalme/cfoundationallib Note: The best way to contact me is through email, not social media. Please feel very free to email me if you want to express feedback, suggest an improvement, desire to collaborate on this free and open source project, want to support me, or want to create something great. Complacency and obstructionism and whining are not tolerated. I desire to make this library the best theoretically possible, so please, let us connect. <h1>This code is in the public domain, fully. You can do whatever you want with it. See docs.html for API reference.  </h1> <h1>Here's some examples of some things you can do easily with Foundationallib.<br><br> <h3>Use it for scripting purposes...</h3> </h1>  <h1>Take control of the Web - in C.<br><br></h1> 

# foundationallib <h2>Finally, a cross-platform, portable, well-designed, secure, robust, maximally-efficient C foundational library &mdash; Making Engineering And Computing Fast, Secure, Responsive And Easy.</h2> <br> <ul class="features-list"> <li><strong>Enables better Engineering Solutions and Security broadly and foundationally where Software Creation or Development or Script Creation is concerned - whether this be on a local, business, governmental or international basis, and makes things easier - and Computing in General.</strong> Don't Reinvent the Wheel - Use Good Wheels - Be Safe And Secure.</li> <br> <li><strong>Enables a free-flowing dynamic computer usage that you need, deserve and should have, simply because you have a computer. With full speed and with robustness. You deserve to be able to use your computer wholly and fully, with proper and fast operations.</strong></li> <br><li><strong>Enables flexibility and power - makes C accessible to the masses (and faster and more secure) with easy usage and strives to bring people up, not degrade the character or actions of people.</strong> This is a fundamental and unequivocal philosophy difference between this library and many subsections of Software Engineering and the mainstream engineering establishment. For instance, in Python, you cannot read a file easily &ndash; you have to read it line-by-line or open a file, read the lines, then close it. With this library, you can efficiently read 10,000 files in one function call. This library gives power. Any common operation, there ought to be a powerful function for.<br><br>We should not bitch around with assembly when we don't want to; we should also have full speed. Some old "solutions" deliver neither, then culturally degrade programmers because their tools are bad - actually, it just degrades programmers, and gives them bad tools. COBOL is an example ...<br><br>Human technology is about empowerment &ndash; people must fight for it to be empowerment, we don't have time to have AI systems kill us because we want to have bad tools and be weak. We must fight.</li> </ul> <br> <ul> <h2>About Foundationallib</h2> <li>→<strong>Cross platform</strong> - works perfectly in embedded, server, desktop, and all platforms - tested for Windows and UNIX - 64-bit and 32-bit, includes a 3-aspect test suite, with more to come.</li> <li>→<strong>Bug free. Reliable. Dependable. Secure. Tested well.</strong></li> <li>→<strong>Zero Overhead</strong> - Only 1 byte due to the power of the error handling, can be configured will full power.</li> <li>→<strong>Static Inline Functions if you want them</strong> (optional) - Eliminating function call overhead to 0 if you wish, for improved performance.</li> <li>→<strong>Custom allocators</strong> - if you want it.</li> <li>→<strong>Custom error handling</strong> - if you want it.</li> <li>→<strong>Safe functions</strong> warn the programmer about NULL values and unused return values. Can be configured to not compile if not Secure. Optional null-check macros in every library function. Does not use any of <code>"gets", "fgets", "strcpy", "strcat", "sprintf", "vsprintf", "scanf", "fscanf", "system", "chown", "chmod", "chgrp", "alloca", "execl", "execle", "execlp", "execv", "execve", "execvp", "bcopy", "bzero"</code>. You can configure it to never use any unsafe functions.</li> <li>→<strong>Portable</strong> - works on all platforms, using platform specific features (using #ifdefs) to make functions better and faster.</li> <li>→<strong>Multithreading support</strong> (optional), with list_comprehension_multithreaded (accepts any number of threads, works in parallel using portable C11 threads)</li> <li>→<strong>Networking support</strong> (optional), using libcurl - making it extremely easy to download websites and arrays of websites - features other languages do not have.</li> <li>→Very good and thorough <strong>Error Handling</strong> and <strong>allocation overflow</strong> checking (good for <strong>Security and Robustness</strong>) in the functions. Allows the programmer to dynamically choose to catch all errors in the functions with a handler (default or custom), or to ignore them. No need to ALWAYS say "if (.....) if you don't want to. Can be changed at runtime.</li> <li>→<strong>Public Domain</strong> so you make the code how you want. (No need to "propitiate" to some "god" of some library).</li> <li>→<strong>Minimal abstractions or indirection of any kind or needless slow things that complicate things</strong> - macros, namespace collision, typedefs, structs, object-orientation messes, slow compilation times, bloat, etc., etc.</li> <li>→<strong>No namespace pollution</strong> - you can generate your <span style=font-style:normal;><b>own version</b></span> with any prefix you like!</li> <li>→<strong>Relies <span style=font-style:normal;>minimally</span> on C libraries - it can be fully decoupled from LIB C and can be statically linked.</strong></li> <li>→<span style=font-style:normal;><b>Very small</b></span> - 13K Lines of Code (including Doxygen comments and following of Best Practices)</li> <li>→<strong>No Linkage Issues or dependency hell</strong></li> <li>→<strong>Thorough and clear documentation</strong>, with examples of usage.</li> <li>→<strong>No licensing restrictions whatsoever - use it for your engineering project, your startup, your Fortune 500 company, your personal project, your throw-away script, your government.</strong></li> <li>→<strong>Makes C like Python or Perl or Ruby in many ways - or more easy</strong></li> <li>→<strong>Easy Straightforward Transpilation Support</strong> - to make current code, much faster - all without any bloat (See transpile_slow_scripting_into_c.rb). <li><h4>In many cases, there is now a direct mapping of functions from other languages into optimized C. See the example script in this repository. This makes optimizing your Python / Perl / Ruby / PHP etc. script very easy, either manually or through the use of AI.</h4></li> </ul> </p> </div> <div class=pane style='border: 0;border-right: 1px dotted rgb(200, 200, 200); background-color: rgb(255, 255, 190);'> <div class="library-details"><h2 style=color:green;>Foundationallib Features</h2> <p class=feature> <strong>Functional Programming Features</strong> - <code>map, reduce, filter,</code> List Comprehensions in C and much more!</p> <p class=feature><strong>Expands C's Primitives for easy manipulation of data types</strong> such as Arrays, Strings, <code>Dict</code>, <code>Set</code>, <code>FrozenDict</code>, <code>FrozenSet</code> - <strong>and enables easy manipulation, modification, alteration, comparison, sorting, counting, IO (printing) and duplication of these at a very comfortable level</strong> - something very, very rare in C or C++, <i>all without any overhead.</i></p> <p class=feature><strong>More comfortable IO</strong> - read and write entire files with ease, and convert complex types into strings or print them on the screen with ease. </p> <p class=feature><strong>A powerful general purpose Foundational Library</strong> - <i>which has anything and everything you need</i> - from <code>replace_all()</code> to <code>replace_memory()</code> to <code>find_last_of()</code> to to <code>list_comprehension()</code> to <code>shellescape()</code> to <code>read_file_into_string()</code> to <code>string_to_json()</code> to <code>string_to_uppercase()</code> to <code>to_title_case()</code> to <code>read_file_into_array()</code> to <code>read_files_into_array()</code> to <code>map()</code> to <code>reduce()</code> to <code>filter()</code> to <code>list_comprehension_multithreaded()</code> to <code>frozen_dict_new_instance()</code> to <code>backticks()</code> - everything you would want to make quick and optimally efficient C programs, this has it.</p> <div style='height: 1px; border: 0;border-bottom: 1px dashed rgb(200, 200, 200);'></div> <p class=performance><span>Helps to make programs hundreds of times faster than other languages with similar ease of creation.</span> <hr> <p class=feature><strong>Easily take advantage of CPU cores with list_comprehension_multithreaded()</strong>.<br><br>You can specify the number of threads, the transform and the filter functions, and this will transform your data - all in parallel. Don't have a multithreaded environment? Then disable it (set the flag).</p> <hr> <h3>You don't want to be reinventing the wheel and hoping that your memory allocation is secure enough - and then failing. <strong>Security Is Paramount.</strong></h3> <h3>You don't want to be waiting <span style='color:rgb(240, 0, 0);'>a day</span> for an operation to complete when it could take <span style='color:rgb(30, 30, 255);'>less than an hour</span>.</h3> <br><p>This library is founded on very strong and unequivocal goals and philosophy. In fact, I have written many articles about the foundation of this library and more relevantly the broader context. See the Articles folder - for some of the foundation of this library.</p> <br><p>This library is an ideal and a dream - not just a Software Library. As such, I would highly suggest that you support me in this mission. Even if it's different from the status quo. Are you a Rust or Zig fan? Then make a Rust or Zig version of this ideal. Let's go. Give me an email.</p> </div> </div> <br> No Copyright - Public Domain - 2023, Gregory Cohen <gregorycohennew@gmail.com> DONATION REQUEST: If this free software has helped you and you find it valuable, please consider making a donation to support the ongoing development and maintenance of this project. Your contribution helps ensure the availability of this library to the community and encourages further improvements. Donations can be made at: https://www.paypal.com/paypalme/cfoundationallib Note: The best way to contact me is through email, not social media. Please feel very free to email me if you want to express feedback, suggest an improvement, desire to collaborate on this free and open source project, want to support me, or want to create something great. Complacency and obstructionism and whining are not tolerated. I desire to make this library the best theoretically possible, so please, let us connect. <pre><code> Mirror Links Blog - https://foundationallib.wordpress.com/ Github - https://github.com/gregoryc/foundationallib Ruby Gem Mirror - https://rubygems.org/gems/foundational_lib Ruby Gem Mirror - https://rubygems.org/gems/foundational_lib2 Library Instagram - https://www.instagram.com/foundationallib Google Drive Mirrors ZIP - https://drive.google.com/file/d/1bK2njCRsH4waTm4LP16sloPQawk7JIR5/view?usp=sharing TAR.GZ - https://drive.google.com/file/d/1RCA1yy9R3cEqI_X9Lv0fxqh-zgNCK005/view?usp=sharing TAR.BZ2 - https://drive.google.com/file/d/1ljdlI_fEnMS_X5WmuhI1qavhgseWlD5j/view?usp=sharing </code></pre> <h1>This code is in the public domain, fully. You can do whatever you want with it. See docs.html for API reference.  </h1> <h1>Here's some examples of some things you can do easily with Foundationallib.<br><br> <h3>Use it for scripting purposes...</h3> </h1>  <h1>Take control of the Web - in C.<br><br></h1> 

README ====== This is a simple API to evaluate information retrieval results. It allows you to load ranked and unranked query results and calculate various evaluation metrics (precision, recall, MAP, kappa) against a previously loaded gold standard. Start this program from the command line with: retreval -l <gold-standard-file> -q <query-results> -f <format> -o <output-prefix> The options are outlined when you pass no arguments and just call retreval You will find further information in the RDOC documentation and the HOWTO section below. If you want to see an example, use this command: retreval -l example/gold_standard.yml -q example/query_results.yml -f yaml -v INSTALLATION ============ If you have RubyGems, just run gem install retreval You can manually download the sources and build the Gem from there by `cd`ing to the folder where this README is saved and calling gem build retreval.gemspec This will create a gem file called which you just have to install with `gem install <file>` and you're done. HOWTO ===== This API supports the following evaluation tasks: - Loading a Gold Standard that takes a set of documents, queries and corresponding judgements of relevancy (i.e. "Is this document relevant for this query?") - Calculation of the _kappa measure_ for the given gold standard - Loading ranked or unranked query results for a certain query - Calculation of _precision_ and _recall_ for each result - Calculation of the _F-measure_ for weighing precision and recall - Calculation of _mean average precision_ for multiple query results - Calculation of the _11-point precision_ and _average precision_ for ranked query results - Printing of summary tables and results Typically, you will want to use this Gem either standalone or within another application's context. Standalone Usage ================ Call parameters --------------- After installing the Gem (see INSTALLATION), you can always call `retreval` from the commandline. The typical call is: retreval -l <gold-standard-file> -q <query-results> -f <format> -o <output-prefix> Where you have to define the following options: - `gold-standard-file` is a file in a specified format that includes all the judgements - `query-results` is a file in a specified format that includes all the query results in a single file - `format` is the format that the files will use (either "yaml" or "plain") - `output-prefix` is the prefix of output files that will be created Formats ------- Right now, we focus on the formats you can use to load data into the API. Currently, we support YAML files that must adhere to a special syntax. So, in order to load a gold standard, we need a file in the following format: * "query" denotes the query * "documents" these are the documents judged for this query * "id" the ID of the document (e.g. its filename, etc.) * "judgements" an array of judgements, each one with: * "relevant" a boolean value of the judgment (relevant or not) * "user" an optional identifier of the user Example file, with one query, two documents, and one judgement: - query: 12th air force germany 1957 documents: - id: g5701s.ict21311 judgements: [] - id: g5701s.ict21313 judgements: - relevant: false user: 2 So, when calling the program, specify the format as `yaml`. For the query results, a similar format is used. Note that it is necessary to specify whether the result sets are ranked or not, as this will heavily influence the calculations. You can specify the score for a document. By "score" we mean the score that your retrieval algorithm has given the document. But this is not necessary. The documents will always be ranked in the order of their appearance, regardless of their score. Thus in the following example, the document with "07" at the end is the first and "25" is the last, regardless of the score. --- query: 12th air force germany 1957 ranked: true documents: - score: 0.44034874 document: g5701s.ict21307 - score: 0.44034874 document: g5701s.ict21309 - score: 0.44034874 document: g5701s.ict21311 - score: 0.44034874 document: g5701s.ict21313 - score: 0.44034874 document: g5701s.ict21315 - score: 0.44034874 document: g5701s.ict21317 - score: 0.44034874 document: g5701s.ict21319 - score: 0.44034874 document: g5701s.ict21321 - score: 0.44034874 document: g5701s.ict21323 - score: 0.44034874 document: g5701s.ict21325 --- query: 1612 ranked: true documents: - score: 1.0174774 document: g3290.np000144 - score: 0.763108 document: g3201b.ct000726 - score: 0.763108 document: g3400.ct000886 - score: 0.6359234 document: g3201s.ct000130 --- **Note**: You can also use the `plain` format, which will load the gold standard in a different way (but not the results): my_query my_document_1 false my_query my_document_2 true See that every query/document/relevancy pair is separated by a tabulator? You can also add the user's ID in the fourth column if necessary. Running the evaluation ----------------------- After you have specified the input files and the format, you can run the program. If needed, the `-v` switch will turn on verbose messages, such as information on how many judgements, documents and users there are, but this shouldn't be necessary. The program will first load the gold standard and then calculate the statistics for each result set. The output files are automatically created and contain a YAML representation of the results. Calculations may take a while depending on the amount of judgements and documents. If there are a thousand judgements, always consider a few seconds for each result set. Interpreting the output files ------------------------------ Two output files will be created: - `output_avg_precision.yml` - `output_statistics.yml` The first lists the average precision for each query in the query result file. The second file lists all supported statistics for each query in the query results file. For example, for a ranked evaluation, the first two entries of such a query result statistic look like this: --- 12th air force germany 1957: - :precision: 0.0 :recall: 0.0 :false_negatives: 1 :false_positives: 1 :true_negatives: 2516 :true_positives: 0 :document: g5701s.ict21313 :relevant: false - :precision: 0.0 :recall: 0.0 :false_negatives: 1 :false_positives: 2 :true_negatives: 2515 :true_positives: 0 :document: g5701s.ict21317 :relevant: false You can see the precision and recall for that specific point and also the number of documents for the contingency table (true/false positives/negatives). Also, the document identifier is given. API Usage ========= Using this API in another ruby application is probably the more common use case. All you have to do is include the Gem in your Ruby or Ruby on Rails application. For details about available methods, please refer to the API documentation generated by RDoc. **Important**: For this implementation, we use the document ID, the query and the user ID as the primary keys for matching objects. This means that your documents and queries are identified by a string and thus the strings should be sanitized first. Loading the Gold Standard ------------------------- Once you have loaded the Gem, you will probably start by creating a new gold standard. gold_standard = GoldStandard.new Then, you can load judgements into this standard, either from a file, or manually: gold_standard.load_from_yaml_file "my-file.yml" gold_standard.add_judgement :document => doc_id, :query => query_string, :relevant => boolean, :user => John There is a nice shortcut for the `add_judgement` method. Both lines are essentially the same: gold_standard.add_judgement :document => doc_id, :query => query_string, :relevant => boolean, :user => John gold_standard << :document => doc_id, :query => query_string, :relevant => boolean, :user => John Note the usage of typical Rails hashes for better readability (also, this Gem was developed to be used in a Rails webapp). Now that you have loaded the gold standard, you can do things like: gold_standard.contains_judgement? :document => "a document", :query => "the query" gold_standard.relevant? :document => "a document", :query => "the query" Loading the Query Results ------------------------- Now we want to create a new `QueryResultSet`. A query result set can contain more than one result, which is what we normally want. It is important that you specify the gold standard it belongs to. query_result_set = QueryResultSet.new :gold_standard => gold_standard Just like the Gold Standard, you can read a query result set from a file: query_result_set.load_from_yaml_file "my-results-file.yml" Alternatively, you can load the query results one by one. To do this, you have to create the results (either ranked or unranked) and then add documents: my_result = RankedQueryResult.new :query => "the query" my_result.add_document :document => "test_document 1", :score => 13 my_result.add_document :document => "test_document 2", :score => 11 my_result.add_document :document => "test_document 3", :score => 3 This result would be ranked, obviously, and contain three documents. Documents can have a score, but this is optional. You can also create an Array of documents first and add them altogether: documents = Array.new documents << ResultDocument.new :id => "test_document 1", :score => 20 documents << ResultDocument.new :id => "test_document 2", :score => 21 my_result = RankedQueryResult.new :query => "the query", :documents => documents The same applies to `UnrankedQueryResult`s, obviously. The order of ranked documents is the same as the order in which they were added to the result. The `QueryResultSet` will now contain all the results. They are stored in an array called `query_results`, which you can access. So, to iterate over each result, you might want to use the following code: query_result_set.query_results.each_with_index do |result, index| # ... end Or, more simply: for result in query_result_set.query_results # ... end Calculating statistics ---------------------- Now to the interesting part: Calculating statistics. As mentioned before, there is a conceptual difference between ranked and unranked results. Unranked results are much easier to calculate and thus take less CPU time. No matter if unranked or ranked, you can get the most important statistics by just calling the `statistics` method. statistics = my_result.statistics In the simple case of an unranked result, you will receive a hash with the following information: * `precision` - the precision of the results * `recall` - the recall of the results * `false_negatives` - number of not retrieved but relevant items * `false_positives` - number of retrieved but nonrelevant * `true_negatives` - number of not retrieved and nonrelevantv items * `true_positives` - number of retrieved and relevant items In case of a ranked result, you will receive an Array that consists of _n_ such Hashes, depending on the number of documents. Each Hash will give you the information at a certain rank, e.g. the following to lines return the recall at the fourth rank. statistics = my_ranked_result.statistics statistics[3][:recall] In addition to the information mentioned above, you can also get for each rank: * `document` - the ID of the document that was returned at this rank * `relevant` - whether the document was relevant or not Calculating statistics with missing judgements ---------------------------------------------- Sometimes, you don't have judgements for all document/query pairs in the gold standard. If this happens, the results will be cleaned up first. This means that every document in the results that doesn't appear to have a judgement will be removed temporarily. As an example, take the following results: * A * B * C * D Our gold standard only contains judgements for A and C. The results will be cleaned up first, thus leading to: * A * C With this approach, we can still provide meaningful results (for precision and recall). Other statistics ---------------- There are several other statistics that can be calculated, for example the **F measure**. The F measure weighs precision and recall and has one parameter, either "alpha" or "beta". Get the F measure like so: my_result.f_measure :beta => 1 If you don't specify either alpha or beta, we will assume that beta = 1. Another interesting measure is **Cohen's Kappa**, which tells us about the inter-agreement of assessors. Get the kappa statistic like this: gold_standard.kappa This will calculate the average kappa for each pairwise combination of users in the gold standard. For ranked results one might also want to calculate an **11-point precision**. Just call the following: my_ranked_result.eleven_point_precision This will return a Hash that has indices at the 11 recall levels from 0 to 1 (with steps of 0.1) and the corresponding precision at that recall level.

No description provided.

No description provided.

No description provided.

No description provided.

No description provided.

No description provided.

No description provided.

No description provided.

No description provided.

No description provided.

No description provided.

No description provided.