blogs.perl.org

Its get on with it day here in the Moose-Pen

After putting on my thinking Moose-cap and reading though some of my notes (and looking at a number of my older posts) the best solution I could come up with was to leave the ability to add a 'JOIN' to an 'UPDATE', 'CREATE' or 'DELETE' statement in.

My reason for this simple if someone wants to use a JOIN is something other than a 'SELECT' and it is perfectly valid command why should my application stop them.

The original DataAccessor code it is written in such a way that you can only do a JOIN on a SELECT as the concept was to have a class that was for 'CUD' actions and one for 'Retrieve'. I would still like something like that and the way I can establish this is with this attribute;

has retrieve_only => (
        is          => 'ro',
        isa         => 'Bool',
        default     => 0,
        traits      => ['MooseX::MetaDescription::Meta::Trait'],
        description => { not_in_DAD => 1 }
    );

It test postette day here in the Moose-Pen.

Today I was rerunning all my test cases to seen which ones where breaking and I got though all of the test on Database::Accessor and was most of the way though Driver::DBI and then I got stuck on '40_joins.t' with this error;

# Expected SQL--> SELECT people.first_name,
people.last_name, people.id, address.street FROM people LEFT JOIN address ON
people.id = address.user_id WHERE people.first_name = ?

# Generated SQL-> SELECT people.first_name, people.last_name, people.id,
address.street FROM people WHERE people.first_name = ?

Hmm, missing the complete JOIN clause there. Now I do not think I made such a basic error as forgetting to call the '_join_clause' sub in the Driver::DBI so the first place I looked was my test and fortunately the test hash was just a little out of date.

Now with the new test hash;

Another test postette here in the Moos-Pen today.

To finish off the '30_where_basic.t' test case I decided to add one more test; the function from hell;

(abs((People.salary + .05) * 1.5))*People.overtime+(abs(People.salary+.05) *2)*People.doubletime)

I diligently set that test up and then when I ran it I got

# Expected SQL--> SELECT people.first_name, people.last_name, people.user_id FROM 
people WHERE ((abs(people.salary + ?) * ?) * people.overtime) + ((abs(people.salary + ?) * ?) * people.doubletime) != ?

# Generated SQL-> SELECT people.first_name First, people.last_name Last, people.user_id "User ID" FROM 
people WHERE ((abs(people.salary + ?) * ?) * people.overtime) + ((abs(people.salary + ?) * ?) * people.doubletime) != ?

the conditional part was correct but I was getting the elements left over from my last test. Opps!

Good old perl I always forget it always passes by references so I am kludging that $in_hash of mine when I change the 'elements' in the last test they stick around until I change them back.

I was thinking of doing a copy of the $in_hash ;

perldoc.pl, the alternative perldoc browser, originally leveraged PostgreSQL full text searching. But as I encountered new issues as well as ones I encountered before, I started working to make the search backend swappable.

There are now three search backends available: PostgreSQL, Elasticsearch, and SQLite using the FTS5 extension. While SQLite is of course the simplest to deploy as it requires no setup, I decided against using it for the main perldoc.pl instance as it does not support skipping stopwords, but the backend is now provided so basic search features can be added to a deployment without setting up a database. Additionally, the application can be deployed without any search backend configured, which will just remove the search box and allow viewing of all pages normally.

It is quick fix postette day here in the Moose-Pen

Yesterday I left off with one fail in my '30_where_basic.t' test case

# Expected SQL--> SELECT people.first_name First, people.last_name Last,
 people.user_id "User ID"
# Generated SQL-> SELECT people.first_name, people.last_name, people.user_id 
FROM people

my alias on these Elements

    [
            {
                name => 'first_name',
                alias=> 'First',
            },
            {
                name => 'last_name',
                view => 'people',
                alias=> 'Last'
            },
            {
                name => 'user_id',
                view => 'people',
                alias=> 'User ID'
            }
        ],

are not being adding into my 'SELECT' SQL. This to me was a little odd as I though I had this covered in the '15_alias.t' test case which check for an SQL like this;

'SELECT sys_users.last_name "Last Name", sys_users.first_name "First Name" FROM people sys_users'

I wonder where the bug is. Using that little trick from the other day;

$utils->sql_param_ok( $in_hash,[ $tests->[8]] );

I isolated my testing to the last test re-ran the test and my error was

Late last year, I blogged about my plans for handling the Iridium flare functionality in the Astro-satpass distribution given that these satellites are being taken out of service in favor of a new design that does not flare.

For some time now, most of the updates to Astro-satpass have been to maintain the canned Iridium status table. This seemed wrong to me, so I have pulled Astro::Coord::ECI::TLE::Iridium out into its own distribution. This is currently available on CPAN. The first distribution of Astro-satpass without this module will be version 0.100, which should be out in a few days.

For people who use one of the CPAN clients, this should be a non-event. I know of no downstream packagers except for ActiveState and MacPorts. Last I looked, ActiveState was just automatically picking up all of CPAN (or not, as the case may be), though if there was no PPM package a CPAN client would work. I have filed a bug with MacPorts describing what is going on. We will see what happens.

Its move along day here in the Moose-Pen

Today I managed to get the time to create '25_exe_array.t' which tests DBI execute array and it went quite well. No real story on that one just with my new 'Test::Utils::sql_param_ok' function I was able to quickly expand it out from just 3 tests to 6 in a few minutes.

I then moved on to '30_where_basic.t' where the idea was to test the basic parts of a 'condition', The basic parts being simple single level predicates mixing elements, functions and expressions.

After creating some 64 tests and giving it a few goes to get rid of the usual typos I ended up with 24 fails out of the 64 and all of them where much like this example;

# Expected SQL--> DELETE FROM people WHERE people.first_name != ? AND 
people.bonus * ? <= ? AND left(people.first_name, ?) != ?
# Generated SQL-> DELETE FROM people WHERE first_name != ? AND bonus * ? 
<= ? AND left(first_name,?) != ?

"I always insisted on commenting my Perl. I never got to the very end of the Camel Book. Not in one reading, anyway. I never experimented with the darker side-effects; three or four separate operations per line was always enough for me. Over time, as my responsibilities moved more to programming, I cut back on the sysadmin tasks. Of course, that didn't stop the Perl use completely--it's amazing how often you can find an excuse to automate a task and how often Perl is the answer. But it reduced my Perl to manageable levels, levels that didn't affect my day-to-day functioning."

Read on: https://opensource.com/article/18/7/confessions-recovering-perl-hacker

Its little bird day here in the Moose-Pen

How does that 'The Be Good Tanyas' song go?

And I sing, the littlest birds sing the prettiest songs …

For us Perl types it should be

And I swear, the littlest typos make the ugliest bugs ...

You will remember that I left off yesterday with this little bug;

Expected SQL--> SELECT people.first_name, people.last_name, people.bonus * abs(?) FROM people
# Generated SQL-> SELECT people.first_name, people.last_name, people.bonus * abs(?),1 FROM people

I was getting ',1' tacked onto the end of some of my generated SQL.

This one proved rather frustrating to track down, and it took much more time than one would expect. I suspected it has something to do with the 'function' part of the '_field_sql' code so I started there but despite my best efforts of adding warnings on each line I could not find it.

Its test test and test again day here in the Moose-pen

Today I added in over 70+ tests into '20_fields.t' and I went though almost every conflagration of fields, params, expression and functions I could think of even the expression from hell from this post.

I did today's work by the book meaning I did all the tests first then I did the debugging. Now I am not going to dump 450 plus lines of hash key value pairs here as that would really piss a few people off and I might loose a reader of two, as if anyone really reads this anyway.

I will just give you what I ran into today. First there was the same bug as yesterday only this time on 'Functions'

Can't locate object method "alias" via package "Database::Accessor::Function" at D:\GitHub\database-accessor-driver-dbi\lib/Database/Accessor/Driver/DBI.pm line 412.

Release SPVM 0.0359

SPVM - Fast array and numeric operation, and provide easy way to C/C++ Binding

0.0359 2018-07-16
  - SPVM::CORE become done native compile
  - add join function
  - fix const assignment bug
  - support list syntax
      my $nums = [(1, 2), (3, 4), (5, 6)];
  - object have body field at offset 0. This will fix alignment bugs.

It is putter about some more day here in the Moose-Pen

I was puttering about today with my tests and I discovered something funny on some implications of SQL and it is to do with that 'AS' when playing with 'alias'. Now I knew it was an optional 'key-word' according to the SQL standard what I did not know is a number of DBs will syntax error when an optional key-word is used. So this SQL

SELECT sys_users.last_name AS last FROM people AS sys_users

is correct to standard there are a number of DBs out there that only this will work

SELECT sys_users.last_name last FROM people sys_users

That makes the above one almost 100% universal that I can tell. I originally wanted that 'AS' in there to make the generated SQL much more readable but I have come to realize that '99.9995%' of the time the end user Database::Accessor will never care what the end SQL is just that it works.

It has been six years since I last mentioned anything about githook-perltidy, a tool for the automatic tidying of Perl and POD files during a Git commit. I rely on it every day, and I still make minor improvements to it, so I thought it worth a quick shout out to others who haven't heard about it or upgraded for a while.

Some CPAN distribution authoring tools come with automatic README generation support. At least in the case of Module::Install::ReadmeFromPod and the various Dist::Zilla plugins I've seen, that generation occurs when Makefile.PL is run. For reasons related to my workflow, Git and GitHub, that timing is too late for me. So I've added a README generation feature to githook-perltidy. See the documentation for details.

One final thing to note. This latest round of work on githook-perltidy was triggered by an unrelated minor issue that a user raised. So don't hesitate to let the author of a tool know when you use something, and that you find it sub-optimal. It might just motivate them to do a bunch of work.

Its fix a but post-ette here in the Moose-Pen.

Yesterday I left off with this error;

Can't locate object method "alias" via package "Database::Accessor::Param" GitHub\database-accessor-driver-dbi\lib/Database/Accessor/Driver/DBI.pm line 412.

So today I figure I better fix it. For once in the Perl world the problem and error message are quite clear. Seem I never did add in the 'Database::Accessor::Roles::Alias' into the ' Database::Accessor::Param' class so that is very easy to fix up.

package 
           Database::Accessor::Param;
        use Moose;
        extends 'Database::Accessor::Base';
++        with qw(Database::Accessor::Roles::Alias');
        has value => (

Just add it in an on the retest I bingo;

… ok 9 - 2 Fields and 2 parama retrieve SQL correct

ok 10 - 2 Fields and 2 parama retrieve params correct
…

So a real sort post-ette for today.

I am giving a short talk on "How to become a CPAN contibutor?" at the The Perl Conference in Glasgow 2018. This is going to be my first ever talk at the European Perl Conference. I have already prepared the first draft of the slides for my talk. I will be doing final cleanup in the next few days. I have also got the hotel and train tickets booked.

For a change, this time I am taking my family with me. The plan is to have Glasgow City tour one day and one day Edinburgh City tour with the family.

Before I was mostly associated with London Perl Workshop for obvious reason as I live in London. Few months ago, I got the opportunity to attend German Perl Workshop 2018 in Gummersbach. I must say, I was very impressed with the way the event was handled by capable team of organisors, specially Jens and Roland.

Hello all, this is a fourth blog post in the Machine learning in Perl series, focusing on the AI::MXNet, a Perl interface to Apache MXNet, a modern and powerful machine learning library.

If you're interested in refreshing your memory or just new to the series, please check previous entries over here: 1 2 3

If you're following ML research then you're probably well aware of two most popular libraries out there, Google's TensorFlow and a relative newcomer to the field but rapidly gaining widespread acceptance, Facebook's PyTorch.

The reason why PyTorch has gained so much ground on TensorFlow is in dynamic nature of that library. TensorFlow started as a static graph library (which is easier to optimize) and PyTorch went with dynamically allocated graphs and NumPy (read PDL) style of programming (with a robust GPU support and auto-differentiation of the gradients) that is as easy to debug as an ordinary Python's code.

Its new test day again in the Moose-Pen

Well starting with the 20_fields.t test case today and this is really just this so far;

my $in_hash = {
    da_compose_only           => 1,
    update_requires_condition => 0,
    delete_requires_condition => 0,
    view                      => { name => 'people' },
    elements => [ { name => 'first_name', }, { name => 'last_name', }, ],
};
my $container = {
    last_name  => 'Bloggings',
    first_name => 'Bill',
};
my $tests = [
    {
        caption => 'Fields',
        create  => {
            container => $container,
            sql =>
              "INSERT INTO people ( first_name, last_name ) VALUES( ?, ? )",
            params => [ 'Bill', 'Bloggings' ]
        },

        retrieve =>
          { sql => "SELECT people.first_name, people.last_name FROM people" },
        update => {
            container => { first_name => 'Robert' },
            sql       => "UPDATE people SET first_name = ?",
            params    => ['Robert']
        },
        delete => { sql => "DELETE FROM people" },
    },
    {
        caption  => '2 Fields and 2 parama',
        key      => 'elements',
        elements => [
            { value => 'User Name:' },
            {
             name => 'first_name',
            },
            { value => 'Address:' },
            { name => 'last_name', },

        ],
        create => {
            container => $container, 
            sql =>
              "INSERT INTO people ( first_name, last_name ) VALUES( ?, ? )",
            params => [ 'Bill', 'Bloggings' ]
        },

        retrieve => {
            sql =>
              "SELECT ?, people.first_name, ?, people.laste_name FROM people",
            params=>[ 'Bill', 'Bloggings']  
        },
        update => {
            container => { first_name => 'Robert' },
            sql       => "UPDATE people SET first_name = ?",
            params    => ['Robert']
        },
        delete => { sql => "DELETE FROM people" },
    },
];
$utils->sql_param_ok($in_hash,$tests);

Which translates into 13 test compared to the seven I had in the previous incarnation of the fields test case.

It carry on testing developing day here in the Moose-Pen.

I left off yesterday with a new Test::Utils->sql_param_ok sub that is going to save a good deal of coding effort as I move forward once I get all the tests up to the new standard of curse, (one foot forward one and a half steps back)

Today I am going to carry on with alias testing this time I am checking basic field alias. To accomplish this I changed the 'elements' in my $in_hash to ones with field aliases;

$in_hash->{elements}= [
        {
            name => 'last_name',
            alias=>  'last'
        },
        {
            name => 'first_name',
            alias=> 'first'
        },
    ];

Next I want to change only the 'retrieve' test a little by update the sql like this

$tests->[0]->{retrieve}->{sql}="SELECT sys_users.last_name AS last, sys_users.first_name AS first FROM people sys_users";

to one that expects the SQL keyword 'AS' followed by the 'alias'. Then I just just the same test again and finally I change the caption of the tests so I can tell them apart and I run my test;

Its go back and fix day here in the Moose-Pen

In a recent post I mentioned that I went out and read though a number of SQL syntax books to make sure I was doing things correctly in which SQL clauses (Where, Sort, Link) goes where. That go me to thinking was the system I was using to test the SQL on a real DB using standard SQL?

Normally I test with Oracle App Express as it work well on a Windows box and has a nice UI but I just noticed that I have some odd SQL in one of my test cases 15_alias.t;

I was starting with this hash;

my $in_hash = {
    da_compose_only=>1,
    update_requires_condition=>0,
    delete_requires_condition=>0,
    view     => { name => 'people',
                  alias=> 'sys_users' },
    elements => [
        {
            name => 'last_name',
        },
        {
            name => 'first_name',
        },
    ],  
};

And from my playing about I would generate this SQL

INSERT INTO people sys_users ( sys_users.first_name, sys_users.last_name ) VALUES( ?, ? )

Our new Perl internship started on the 9th of July, 7 daring new interns took on the challenge of becoming Perl developers. The internship will last for 4 weeks, during which they will get familiar with Perl and all that goes with it.

Perl is one of the pillars of Evozon, the company was founded in 2005 as a Perl and Java shop, expanding to other technologies over the years. After 13 years of Perl development we have plenty of experience and excitement for this language, something we want to share with this new group of interns.

We have one of the largest Perl teams in Europe and we’re proud to say that quite a few of our developers started their career with us, learning about Perl through internships just like this one. Now, they’re passing on the knowledge they’ve accumulated over the years to another round of Perl developers to be.

The internship will have two parts, a theoretical part where the interns will be introduced to the world of Perl by our trainers and a practical part where they will be working on an application built in Mojolicious with DBIx and MySQL, using the microcontroller Raspberry Pi. The microcontroller will have several types of sensors attached and will be able to monitor the temperature and humidity in our office.