Data::Consumer Data::Consumer - repeatedly consume a data resource in a robust way.
Download

Data::Consumer Ranking & Summary

Data::Consumer Tags

Perl module resource data data resource data consumer consumer

Data::Consumer Description

Data::Consumer - repeatedly consume a data resource in a robust way. Data::Consumer - repeatedly consume a data resource in a robust way.SYNOPSIS use Data::Consumer; my $consumer = Data::Consumer->new( type => $consumer_name, unprocessed => $unprocessed, working => $working, processed => $processed, failed => $failed, max_passes => $num_or_undef, max_process => $num_or_undef, max_elapsed => $seconds_or_undef, ); $consumer->consume( sub { my $id = shift; print "processed $idn"; } );It is a common requirement to need to process a feed of items of some sort in a robust manner. Such a feed might be records that are inserted into a table, or files dropped in a delivery directory. Writing a script that handles all the edge cases, like getting "stuck" on a failed item, and manages things like locking so that the script can be parallelized can be tricky and is certainly repetitive.The aim of Data::Consumer is to provide a framework to allow writing such consumer type scripts as easy as writing a callback that processes each item. The framework handles the rest.The basic idea is that one need only use, or in the case of a feed type not already supported, define a Data::Consumer subclass which implements a few reasonably well defined primitive methods which handle the required tasks, and then the Data::Consumer methods use those to provide a DWIMily consistant interface to the end consumer.Currently Data::Consumer is distributed with two subclasses, Data::Consumer::MySQL for handling records in a MySQL db (using the MySQL GET_LOCK() function), and Data::Consumer::Dir for handling a drop directory scenario (like for FTP or a mail directory).Once a resource type has been defined as a Data::Consumer subclass the use pattern is to construct the subclass with the appropriate arguments, and then call consume with a callback. Requirements: · Perl

Data::Consumer Related Software