Async Queue implementation without locking I wrote a quick implementation of an async queue that utilizes a backing ConcurrentQueue. It was pretty much based on an implementation given in this Stack Overflow answer. The only difference is that I got rid of the internal locks that implementation uses.

Since a ConcurrentQueue is already thread safe, I can't figure out why they chose to use a lock in their implementation. Normally I would ask them on Stack Overflow but the question is 4 years old and I would rather post my implementation here and have someone do a peer review to see if I have a potential threading issue.

/// <summary>
///     Asynchronous Queue.
/// </summary>
/// <typeparam name="T">
///     The queue's elements' type.
/// </typeparam>
public sealed class AsyncQueue<T> {
    /// <summary>
    ///     Items.
    /// </summary>
    private readonly ConcurrentQueue<T> _items;

    /// <summary>
    ///     Promises.
    /// </summary>
    private readonly ConcurrentQueue<TaskCompletionSource<T>> _promises;

    /// <summary>
    ///     Create an Asynchronous Queue.
    /// </summary>
    public AsyncQueue() {
        this._items = new ConcurrentQueue<T>();
        this._promises = new ConcurrentQueue<TaskCompletionSource<T>>();
    }

    /// <summary>
    ///     Dequeue an Item Asynchronously.
    /// </summary>
    /// <returns>
    ///     A task representing the asynchronous operation.
    /// </returns>
    public Task<T> DequeueAsync() {
        var dequeueTask = this.DequeueAsync(CancellationToken.None);
        return dequeueTask;
    }

    /// <summary>
    ///     Dequeue an Item Asynchronously.
    /// </summary>
    /// <param name="cancellationToken">
    ///     A cancellation token to cancel the asynchronous operation with.
    /// </param>
    /// <returns>
    ///     A task representing the asynchronous operation.
    /// </returns>
    public async Task<T> DequeueAsync(CancellationToken cancellationToken) {
        CancellationTokenRegistration? cancellationTokenRegistration = null;
        var promise = new TaskCompletionSource<T>();
        var itemFound = this._items.TryDequeue(out var item);
        if (!itemFound) {
            cancellationTokenRegistration = cancellationToken.Register(OnCancellationTokenCanceled, promise);
            this._promises.Enqueue(promise);
        }
        else {
            promise.TrySetResult(item);
        }

        try {
            item = await promise.Task;
            return item;
        }
        finally {
            cancellationTokenRegistration?.Dispose();
        }

        // <summary>
        //      On Cancellation Token Canceled.
        // </summary>
        void OnCancellationTokenCanceled(object cState) {
            var cPromise = (TaskCompletionSource<T>) cState;
            cPromise.TrySetCanceled();
        }
    }

    /// <summary>
    ///     Enqueue an Item.
    /// </summary>
    /// <param name="item">
    ///     An item to enqueue.
    /// </param>
    public void Enqueue(T item) {
        while (true) {
            var promiseFound = this._promises.TryDequeue(out var promise);
            if (!promiseFound) {
                this._items.Enqueue(item);
                break;
            }

            var promiseSet = promise.TrySetResult(item);
            if (promiseSet) {
                break;
            }
        }
    }
}

I wrote unit tests and it seems to work fine. I think there might be a rare race condition that happens in the following scenario, but I have not been able to trigger it and I want a second opinion:

Thread 1 enqueues an item
Thread 1 does not find an existing promise
Thread 1 is preempted
Thread 2 attempts to dequeue an item
Thread 2 does not find an existing item
Thread 2 is preempted
Thread 1 enqueues item
Thread 2 creates and enqueues a promise
Idle time until another enqueue or dequeue operation occurs

The only difference is that I got rid of the internal locks that implementation uses.

  Since a ConcurrentQueue is already thread safe, I can't figure out why they chose to use a lock in their implementation.

From my understanding of the meaning of "thread safety":

  All public and protected members of ConcurrentQueue<T> are thread-safe and may be used concurrently from multiple threads.

A class being "thread safe" just means its members are, but only individually. So one method being called from many threads will not cause problems. Interacting with many members from many threads might not be "thread safe".

In some sense, the meaning of "thread safety" depends on your application.

If you look at the lock from the answer:

lock (_syncRoot)
{
    if (_promisesQueue.TryDequeue(out promise) &&
        !promise.Task.IsCanceled &&
        promise.TrySetResult(item))
    {
        return;
    }
    _bufferQueue.Enqueue(item);
}

All those statements in the block are combined together to be accessed only by one thread, which is something the "thread safety" of the class alone cannot guarantee.

The lock allows you to group code into a thread-safety-wise "atomic" block.


Find the minimum number of characters to add to make password strong This is the criteria:

Its length is at least 6. It contains at least one digit. It contains at least one lowercase English character. It contains at least one uppercase English character. It contains at least one special character. The special characters are: !@#$%^&*()-+

My code is working for all test cases but I'm afraid it seems way longer than necessary. I'm trying to trim it down.

import math
import os
import random
import re
import sys

# Complete the minimumNumber function below.
def minimumNumber(n, password):
    # Return the minimum number of characters to make the password strong
    flag = 0
    if len(password) >= 6:
        passRegex = re.compile(r"^(?=.*[\d])")
        mo = passRegex.search(password)
        if mo is None:
            flag+=1
        passRegex = re.compile(r"^(?=.*[A-Z])")
        mo = passRegex.search(password)
        if mo is None:
            flag+=1
        passRegex = re.compile(r"^(?=.*[a-z])")
        mo = passRegex.search(password)
        if mo is None:
            flag+=1
        passRegex = re.compile(r'[!@#$%^&*()-+]')
        mo = passRegex.search(password)
        if mo is None:
            flag+=1
        return flag
    else:
        if any(x in "0123456789" for x in password) is False:
            flag+=1
        if any(x in "ABCDEFGHIJKLMNOPQRSTUVWXYZ" for x in password) is False:
            flag+=1
        if any(x in "abcdefghijklmnopqrstuvwxyz" for x in password) is False:
            flag+=1
        if any(x in "!@#$%^&*()-+" for x in password) is False:
            flag+=1
        if (flag+len(password)<6):
            return 6-len(password)
        if (flag+len(password)>=6):
            return flag

if __name__ == '__main__':

    n = int(input())

    password = input()

    answer = minimumNumber(n, password)

Any advice is appreciated.

Code length is one factor in readability, but it isn't the only factor. "Is this code easy to read and modify" is a more important question to ask.

It looks like you've gotten a method skeleton here, which kind of leads you down the path of stuffing everything in one function, which is a bit of a shame. Let's look at what we actually need here.

Length of at least 6
One lowercase English character
One uppercase English character
One digit
One special character

What if we split this up into a bunch of separate functions, which each do one thing. We'll just write the function names down for now, and leave the implementation blank.

def lower_required(text):
    pass

def upper_required(text):
    pass

def digits_required(text):
    pass

def length_required(text):
    pass

def special_required(text):
    pass

Now, Python has a string module that is (semi)deprecated (much of what was in it is now on string objects themselves), but that has some still useful things, namely, string.ascii_lowercase, string.ascii_uppercase, and string.digit. We can use these in the implementations for some of our functions. Let's just write them as simply as possible for now.

import string

def lower_required(text):
    for char in text:
        if char in string.ascii_lowercase:
            return 0
    return 1

def upper_required(text):
    for char in text:
        if char in string.ascii_uppercase:
            return 0
    return 1

def digit_required(text):
    for char in text:
        if char in string.digits:
            return 0
    return 1

There's quite a bit of repetition here that we can factor out.

def num_required(text, characters):
    if any((x in characters for x in text)):
        return 0
    return 1

Then our previous functions can just call this:

def lower_required(text):
    return num_required(text, string.ascii_lowercase)

And similarly for the upper_required and digit_required.

In fact, we can do this with our special characters as well:

SPECIAL_CHARS = '!@#$%^&*()-+'

def special_required(text):
    return num_required(text, SPECIAL_CHARS)

The last function we need to implement is length_required:

MINIMUM_LENGTH = 6

def length_required(text):
    if len(text) >= MINIMUM_LENGTH:
        return 0
    return MINIMUM_LENGTH - len(text)

Now we can stitch these all together within minimumNumber (note that Python functions should be named using snake_case, so it should be minimum_number here - but I assume this function name was given to you, so I won't harp on about it too much).

def minimumNumber(text):
    test_functions = [lower_required, upper_required,
                      digits_required, special_required]
    min_chars_required = sum([test_fn(text) for test_fn in test_functions])
    missing_length = length_required(text)
    if missing_length > min_chars_required:
        return missing_length
    return min_chars_required

The benefit of doing this is the fact that we can easily add extra checks. What if a new requirement comes along? We can just create a new function an add it to test_functions; the amount of existing code we have to touch is minimal.

Just to drive home the point about size vs readability, here is a (slightly) code golfed version:

import string

def minimumRequired(password):
    if len(password) < 6:
        return 6 - len(password)
    unique_pw = frozenset(password)
    char_sets = (frozenset(string.ascii_lowercase), frozenset(string.ascii_uppercase),
                 frozenset(string.digits), frozenset('!@#$%^&*()-+'))
    required_extra = 0
    for char_set in char_sets:
        if not char_set & unique_pw:
            required_extra += 1
    return required_extra

This (probably - I haven't really tested it) satisfies the requirements, but how do I make changes to it? Is it obvious how it does what it does (I'd argue not really).


Process files in a directory and upload them to URL I have some code that I wrote for a personal project which takes, as arguments, a directory and a URL. It then processes the files in the named directory, archives them, and uploads the archive to the specified URL.

I have heard that there are many ways to do this(TIMTOWTDI), particularly, with globs. I am hesitant to change working code, but I am interested in possible optimizations that may be possible with "globbing".

What can I do to make my code more idiomatic?

#!/usr/bin/perl

use strict;
use warnings;
use Data::Dumper qw(Dumper);
use File::Spec qw(catfile rel2abs);
use Digest::SHA qw(sha256_hex);
use Archive::Tar;
use JSON;
use HTTP::Request::Common;
use LWP::UserAgent;

my $dir = $ARGV[0];
my $url = $ARGV[1];
my $fullpath;
my $fullsize;
my $fullid;
my @AoH;
my @checksumfiles;
my $archive = "archive.tar.gz";

opendir DIR, $dir or die "cannot open dir $dir: $!\n";
chdir $dir or die "cannot navigate to dir $dir: $!\n";

while(my $file = readdir DIR) {
    next unless(-f File::Spec->catfile($dir, $file));
    next if($file =~ m/^\./);

    $fullpath = File::Spec->rel2abs($file);
    $fullsize = -s File::Spec->catfile($dir, $file);
    $fullid   = sha256_hex($fullpath);

    my %hash = (
        path    =>  $fullpath,
        size    =>  $fullsize,
        id      =>  $fullid,
    );

    push(@AoH, \%hash);
}
my @array;
for my $i(0..$#AoH) {
    no warnings 'uninitialized';
    my ($ext) = $AoH[$i]{path} =~ (/(\.[^.]+)$/);
    my $idext = $AoH[$i]{id} . $ext;
    push(@checksumfiles, $idext);
    push(@array, $AoH[$i]{path});
}

my $manifest = JSON->new->pretty->encode(\@AoH);

my $filename = "manifest.json";
open my $fh, '>', $filename or die "could not open file $filename: $!\n";
print $fh $manifest;
close $fh;

my $tar = Archive::Tar->new;
$tar->add_files(@array);
$tar->write($archive);
my @file_objs = $tar->read($archive);

for my $i(0..$#file_objs) {
    $file_objs[$i]->rename($checksumfiles[$i]);
}

$tar->add_files($manifest);
$tar->write($archive, COMPRESS_GZIP);

my $browser = LWP::UserAgent->new;
my $useragent = LWP::UserAgent->new(
    env_proxy   => 1,
    keep_alive  => 1,
    timeout     => 120,
    agent       => 'Mozilla/5.0',
);

my $req = POST $url,
    Content_Type    => 'form-data',
    Content         => [ pageAction => 'upload', file => [$archive] ];

$useragent->request($req);

#print Dumper sort \@array;
#print Dumper sort \@checksumfiles;
#print Dumper sort \@AoH;

Your code is already pretty good, you have to tighten a scope of lexical (my) variables. Also as non-perl related, always get rid of unnecessary loops.

use strict;
use warnings;
use Data::Dumper qw(Dumper);
use File::Spec qw(catfile rel2abs);
use Digest::SHA qw(sha256_hex);
use Archive::Tar;
use JSON;
use HTTP::Request::Common;
use LWP::UserAgent;

my ($dir, $url) = @ARGV;
my $archive = "archive.tar.gz";

my @AoH;
my @checksumfiles;
my @array;
# same as glob("$dir/*")
while (my $file = <$dir/*>) {
    -f $file or next;

    my $fullpath = File::Spec->rel2abs($file);
    my %h = (
        path    =>  $fullpath,
        size    =>  -s $file,
        id      =>  sha256_hex($fullpath),
    );
    push @AoH, \%h;

    my ($ext) = $h{path} =~ /(\.[^.]+)$/;
    # defined or
    my $idext = $h{id} . $ext // ''; # fix codereview coloring */
    push @checksumfiles, $idext;
    push @array, $h{path};
}

# my @array;
# for my $i (0..$#AoH) {
    # no warnings 'uninitialized';
    # my ($ext) = $AoH[$i]{path} =~ (/(\.[^.]+)$/);
    # my $idext = $AoH[$i]{id} . $ext;
    # push(@checksumfiles, $idext);
    # push(@array, $AoH[$i]{path});
# }


C++ program to remove duplicate characters from string This is a question in the book "Cracking The Coding Interview".

  Design an algorithm and write code to remove the duplicate characters in a string
  without using any additional buffer. NOTE: One or two additional variables are fine.
  An extra copy of the array is not.

  Follow Up

  Write the test cases for this method.  The four test cases are

  String does not contain any duplicates, e.g.: abcd
  String contains all duplicates, e.g.: aaaa
  Null string
  String with all continuous duplicates, e.g.: aaabbb

I want to optimise this code and whether I can use some other function from STL.

#include <iostream>
#include <string>
#include <algorithm>

std::string & removeDuplicate(std::string& str)
{
  int length = str.length();
  for(unsigned int i = 0; i < length; i++)
  {
    char currChar = str[i]; //holds current character
    for(unsigned int j = i+1; j < length; j++)
    {
      if(currChar == str[j])
        str.erase (std::remove(str.begin()+j, str.end(), str[j]), str.end());
    }
  }
  return str;
}

int main()
{
  std::string str;
  std::cout << "Enter string \n";
  std::getline(std::cin, str);
  str = removeDuplicate(str);
  std::cout <<"New String is " << str << "\n";
}

From the tests we are if we are only looking for repeating chars, not duplicate e.g. we can transform "ABBA" into "ABA"  (and not "AB"). Then we can use something like:

void removeRepeatingChars(char* s){
  if(!s[0]){                  // Check for enpty string
    return;                   // Notning to do per definition
  }

  int last=0;                 // Store position of last seen character  thats not a duplicate

  for(int i = 1; s[i] ; i++){ // Loop over all characters and
    if(s[i] != s[last]){      // check if current character is not a dupllicate.
      last++;                 // If so increment the position of last non duplicate and
      s[last] = s[i];         // copy the currentcharacter to this position.
    }                         // Else go to next char.
  }
  s[++last] = '\0';           // Add string terminator at the end.
}

If not you need to store each seen character and add this check so we get something like.

for(int i = 1; s[i] ; i++){
  if(s[i] != s[last] && !sceen[s[i]]){
      last++;
      s[last] = s[i];
      seen[s[i]] = true;
  }
}

Observe that we use that we can modify the string how ever we like on chars we already have checked in order to do the whole operation in place.


Stepping Number Solution Optimization I have the following problem:

  The stepping number:

  A number is called a stepping number if every adjacent digits, except those separated by commas, differ by 1. A stepping number can't be a 1-digit number, it must be at least a 2-digit number. For example, 45 and 8,343,545 are stepping numbers.
  But, 890,098 is not. The difference between ‘9’ and ‘0’ should
  not be considered as 1.

  Given start number s and an end number e your function should list
  out all the stepping numbers in the range including both the numbers s
  & e.

My Attempt:

public void steppingNumber(int s, int e) {
    while(s <= e) {
        String str = String.valueOf(s);
        if(isSteppingNumber(str)) System.out.print(str + " ");
        s++;
    }
}

public boolean isSteppingNumber(String str) {
    if(str.length() == 1) return false; // 1-digit number can't be a stepping number

    List<String> numbers = new ArrayList<>();

    while(str.length() >= 3) { // get every 3-digit comma-separated number
        numbers.add(str.substring(str.length()-3));
        str = str.substring(0,str.length()-3);
    }
    numbers.add(str); // Also get the last number left

    for(String num : numbers) { // for every 3-digit comma-separated number, check if it's a stepping number
        for(int i = 1; i < num.length(); i++) {
            int previousDigit = Character.getNumericValue(num.charAt(i-1));
            int currentDigit = Character.getNumericValue(num.charAt(i));

            if(Math.abs(previousDigit - currentDigit) != 1) return false;
        }
    }

    return true;
}

If the question were only to check if a number was a stepping number, I think my solution would be fine. However, if I should list all stepping numbers within the range, say 1 to 10^15, then my solution will run linear time, leave alone the checking part. Can anyone give a better solution for the given problem?

public void steppingNumber(int s, int e) {

As described in the problem statement, s is the starting number and e is the ending number.  Why not call them start and end then?  That way we don't have to refer to the problem statement to know what the variables mean.

I would also pick another name for the function.  You are printing the stepping numbers, so call it printSteppingNumbers.  As a general rule, functions should have verb names.  They do things and their names should reflect that.

    String str = String.valueOf(s);

Why a string?  In C, this would make sense as the difference between a letter and a number in C is trivial (char is an integer type).  Java isn't as forgiving.  Even if you do want a string, this is the wrong place to do it.  Call the function with s (or start).

    if ( isSteppingNumber(start) ) {
        System.out.print(start + " ");
    }

I added braces ({}) around the then clause.  This helps avoid a class of error where someone intends to add a new statement to the then clause but actually makes it run whether the if triggers or not.  I also find it easier to read.

List<String> numbers = new ArrayList<>();

while(str.length() >= 3) { // get every 3-digit comma-separated number
    numbers.add(str.substring(str.length()-3));
    str = str.substring(0,str.length()-3);
}

This seems overly complicated.  You don't need to generate a List here.  It's sufficient to check each chunk as you go through.

while ( str.length() > 3 ) {
    if ( ! isSteppingSequence(str.substring(str.length()-3)) ) {
        return false;
    }
    str = str.substring(0,str.length()-3);
}

return isSteppingSequence(str);

You also don't need >= here.  A simple > is better, as you always want there to be one chunk left to test outside the loop.

But we can do better if we keep it as a number at this point.  First, we need a constant.

private final static int CHUNK_SIZE = 1000;

Now we can use it.

    while ( number > CHUNK_SIZE ) {
        if ( ! isSteppingSequence(number % CHUNK_SIZE) ) {
            return false;
        }

        number /= CHUNK_SIZE;
    }

    return isSteppingSequence(number);

Another constant.

private final static int BASE = 10;

Now we just need to define isSteppingSequence and we get:

private final static int BASE = 10;
private final static int CHUNK_SIZE = 1000;

private static void listSteppingNumbers(int start, int end) {
    while ( start <= end ) {
        if ( isSteppingNumber(start) ) {
            System.out.print(start + " ");
        }

        start++;
    }
}

private static boolean isSteppingNumber(int number) {
    if ( number < BASE ) {
        return false;
    }

    while ( number > CHUNK_SIZE ) {
        if ( ! isSteppingSequence(number % CHUNK_SIZE) ) {
            return false;
        }

        number /= CHUNK_SIZE;
    }

    // all the other sequences were stepping, so
    // if this one is, it's a valid stepping number
    // otherwise not
    return isSteppingSequence(number);
}

private static boolean isSteppingSequence(int number) {
    int previous_digit = number % BASE;
    number /= BASE;
    while ( number > 0 ) {
        int current_digit = number % BASE;

        if ( Math.abs(previous_digit - current_digit) != 1 ) {
            return false;
        }

        previous_digit = current_digit;
        number /= BASE;
    }

    // if number was less than BASE, then it skipped the while loop
    // and all single digit numbers are valid stepping sequences
    // so true
    // if it made it through the while loop,
    // all the adjacent digits differed by 1
    // so true
    return true;
}

So we now have a more readable version of your initial algorithm.  Since it doesn't convert to a String and from characters, I suspect that it will be slightly faster.

You also ask if we can do better than stepping through every number in the range and testing to see if it is a stepping number.  Certainly we can.  We can generate the stepping numbers by composition.  We don't have to search for them.  I notice that this is described in more detail on Stack Overflow.


Data encryption / decryption on Node.js I've implemented my first data encryption / decryption methods on Node.js.
Although it might look similar to hundreds of sample implementations available online, I'm curious to get a feedback.

I'm particularly interested in:

some flaw points, potential security, and performance issues
In encrypt() I return an ENCRYPTED object with encrypted data after being freezed by Object.freeze(), are there any drawbacks/pitfalls of returning a freezed object besides the read-only status of the object?

const CRYPTO = require("crypto");

const ALG_CIPHER = "aes-256-gcm";

const ENC_IN = "utf8";
const ENC_OUT = "base64";

const BUFFER_SIZE = 16;

const KEY = Buffer.from("…", ENC_IN);

/**
 * Encrypt data using an initialisation vector
 *
 * @param {string} data - data to be encrypted
 * @returns {Promise<{authTag: Buffer, data: string, iv: Buffer}>} encrypted - encrypted data with decryption parameters
 */
const encrypt = async function encrypt(data) {

    const IV = Buffer.from(CRYPTO.randomBytes(BUFFER_SIZE));
    const CIPHER = CRYPTO.createCipheriv(ALG_CIPHER, KEY, IV);

    let enc = CIPHER.update(data, ENC_IN, ENC_OUT);
    enc += CIPHER.final(ENC_OUT);

    const ENCRYPTED = Object.freeze({
        data: enc,
        iv: IV,
        authTag: CIPHER.getAuthTag()
    });

    return ENCRYPTED;

};

/**
 * Decrypt data using an initialisation vector
 *
 * @param {string} data - data to be decrypted
 * @param {string} iv - initialisation vector
 * @param {Buffer} authTag - authentication tag
 * @returns {Promise<string>} decrypted - decrypted data
 */
const decrypt = async function decrypt(data, iv, authTag) {

    const DECIPHER = CRYPTO.createDecipheriv(ALG_CIPHER, KEY, iv);
    DECIPHER.setAuthTag(authTag);

    let decrypted = DECIPHER.update(data, ENC_OUT, ENC_IN);
    decrypted += DECIPHER.final(ENC_IN);

    return decrypted;

};

Comments are below the code fragments.

const ENC_IN = "utf8";
const ENC_OUT = "base64";

With the choice of encryption and encoding, I'd include the "encoding" part fully (e.g. ENCODING_IN and ENCODING_OUT).

const KEY = Buffer.from("…", ENC_IN);

No, a key should not be a constant, and a key should certainly not be a string, even if it has a specific encoding. A key should be 16, 24 or 32 fully unpredictable bytes (for 128, 192 or 256 bit keys respectively). With UTF-8 you only have some 108 possible printable characters for each byte. Besides that, many libraries will not handle partial or overly long keys well.

If you want to use a password, then use a password based key derivation function (PBKDF) and then createCipheriv / createDecipheriv. Support for that is build in (note that createCipher / createDecipher is deprecated).

Encryption

const encrypt = async function encrypt(data) {

At this point you may want to document the character encoding and the base 64 encoding that is being applied, the mode of operation and that an IV and authentication tag is added. Document or point to the protocol in other words.

const IV = Buffer.from(CRYPTO.randomBytes(BUFFER_SIZE));

Good, random IV. For GCM though I'd use 12 bytes or the mode has to perform additional operations for no security benefit.

const ENCRYPTED = Object.freeze({

Even though you froze the variable, it's still a local variable and therefore you should not be using all uppercase.

data: enc,
iv: IV,
authTag: CIPHER.getAuthTag()

Now this is a bit weird. Unless I'm mistaken enc is now base 64 encoded, but iv and authTag are just plain buffers of binary data.

Decryption

 * @param {string} iv - initialisation vector

Strange, your iv of type Buffer seems to have magically become a string...

Otherwise the decrypt function is nicely symmetrical to the encrypt function, which is good.

Beware of overly stringifying your code. Beware of wrapper classes; don't start using above class for all your code: use user specific encryption routines insetead, and clearly specify your protocol.

When using GCM, you may want to allow additional authenticated data (AAD) to be included in the calculation of the tag.


Reading genetic data in VCF and Tabix formats using an asynchronous library I'm working with an open-source library for processing and parsing genetic data in VCF and Tabix formats. It contains functions and classes that make it easy to read an index file (a Tabix) and load corresponding data from a compressed VCF.

It works asynchronously (unsurprisingly), however, and I'm not sure of the best way to structure my code. I'm looking to make multiple calls to the getRecords() method—since I need to retrieve multiple discontinuous segments of data; because this is asynchronous, it seems like I also have to instantiate a new file reading object (here known as readBinaryVCF but roughly analogous to HTML's FileReader). Both the file reading object's constructor and the getRecords() function use callbacks.

I'm not really experienced in JavaScript or asynchronous design patterns; I'm hoping it doesn't have to be this convoluted. Other solutions that I tried, though, like recursing or making multiple calls  to getRecords() on the same object, were not as flexible or as robust—or just flat out didn't work.

The code sample below is the first iteration I've gotten to work. That being said, please tear it apart. I suspect there're many improvements to be made.

EDIT: I'll leave this code alone as my first draft, but please look at the minor changes I've made in an answer below. I'm still hoping for help, preferably on top of my revisions.

var regions = [
    [10, 10000, 100000],
    [12, 10000, 100000],
    [14, 10000, 100000],
    [18, 10000, 100000]
];

var d = [];
var readers = [];

for (var a = 0; a < regions.length; a++) {
    d.push([]);
    readers.push(new readBinaryVCF(tabix, vcf, function(x) {x = x;}));
    //Should I just initialize these to null instead?
}

regions.forEach(function(item, i) {

    readers[i] = new readBinaryVCF(tabix, vcf, function(tabixReader) {

        rbvcf = readers[i];

        var index = 0;
        for (key in tabixReader.idxContent.namehash) {
            if (key == regions[i][0]) {
                index = tabixReader.idxContent.namehash[key];
            }
        }

        rbvcf.getRecords(index, regions[i][1], regions[i][2], function(data) {
            d[i] = data;

            if (i == regions.length - 1) {
                var finalText = "";

                for (var j = 0; j < d.length; j++) {
                    for (var k = 0; k < d[j].length; k++) {
                        finalText = finalText + d[j][k] + "<br />";
                    }
                }

                $(".text").html(finalText);
            }
        });
    });
});

A few painfully obvious edits of my own:

The frivolous for-loop:

for (var a = 0; a < regions.length; a++) {
    d.push([]);
    readers.push(new readBinaryVCF(tabix, vcf, function(x) {x = x;}));
}

Why, why did I think this was a good idea? It's entirely unnecessary.

Omitted.

Callback-waiting:

if (i == regions.length - 1) {

   var finalText = "";

   for (var j = 0; j < d.length; j++) {
       for (var k = 0; k < d[j].length; k++) {
           finalText = finalText + d[j][k] + "<br />";
       }
   }

   $(".text").html(finalText);
}

I thought I could wait until the "last" callback had ended before parsing my data, not realizing that—because they're, again, callbacks—they won't finish in order. This answer inspired me to use a manual counter to determine when they've all finished.

Revised code:

var d = [];
var readers = [];
var callbacksRemaining = regions.length;

regions.forEach(function(item, i) {

    readers[i] = new readBinaryVCF(tabix, vcf, function(tabixReader) {

        reader = readers[i];

        var index = 0;

        for (key in tabixReader.idxContent.namehash) {
            if (key == regions[i][0]) {
                index = tabixReader.idxContent.namehash[key];
            }
        }

        reader.getRecords(index, regions[i][1], regions[i][2], function(data) {;

            d[i] = data;
            callbacksRemaining--;

            if (callbacksRemaining <= 0) {
                processData(d);
            }
        });
    });
});

Summary:

I've made these few minor edits, but my code still isn't running as fast as I'd like. I'm sure there are many more changes I could and should make—I'm still hoping for advice!


Minimum coins algorithm in Ruby I am working on a puzzle to find the minimum number of coins needed to buy a product.

I am given three coins:

1
3
5

and also a number representing the price of a product:

11

The answer here would be three, since it takes at least three coins to reach this number.

5, 5, 1

I decided it was best to use Ruby's repeated_permutation method. I tested my method in irb and it seems to work for smaller inputs, but when I submitted it to the online grader, I got a Memory Allocation Error.

File.open(ARGV[0]).each_line do |line|
    number = line.to_i
    coins = [1,3,5]
    coin_combos = []
    i = 1
    until coin_combos.flatten(1).any? { |c| c.inject(:+) >= number }
        coin_combos << coins.repeated_permutation(i).to_a
        i += 1
    end
    puts i - 1
end

Here was my thought process:

Store the different types of coins in a coins array
Initialize an empty array for storing sequences of coins
Use a loop. It will be finished once the sum of one of the sequences is equal to or surpasses the product's price.
Push to my array the number of repeated permutations of length equal to my counter
Increment my counter and test for the condition again

This seems to be a problem with my algorithm and I'm using up too much space here.

Is there a different algorithm I should use?

I don't see why you need to calculate permutations. Simply calculate the quotient and remainder of the division starting from the highest coins. In imperative style:

def get_num_coins(coins, value)
  ncoins = 0
  coins.sort.reverse.each do |coin|
    ncoins +=  value / coin
    value = value % coin
  end
  ncoins
end

In functional style:

def get_num_coins(coins, value)
  coins.sort.reverse.reduce(value: value, ncoins: 0) do |state, coin|
    q, r = state[:value].divmod(coin)
    {value: r, ncoins: state[:ncoins] + q}
  end.fetch(:ncoins)
end


PHP's array_diff in JavaScript I shall make a JavaScript-function which does the same as PHP's array_diff:
http://php.net/manual/de/function.array-diff.php

Because it is intended to use the function as a (internal) library-function I have tried to catch everything which could possibly go wrong concerning incorrect usage.
Moreover I've tried to document the usage of the function well.

Can one expect that my algorithm works as expected?

How can I improve my documentation and value-checking further?

Should I use different error return-values to signal more precise what has gone wrong?

// ----- TESTING -------------------------------
var arr1 = [1, 3, 2, 1, 2, 7, 9, 2, 6, 5 ];
var arr2 = [1, 9, 6, 11, 0, 17, 4, 22];

var str1 = [1, 9, '6', 11, 0, 17, 4, 22];
var str2 = [1, 9, '6', 11, 'zero', 17, 4, 22];

var not = [1, 9, NaN, 11, 0, 17, 4, 22];
var flt = [1, 9, 2.68, 11, 0.32, 17, 4, 22];

var e1 = {};
var e2 = 31;

console.log(getArrayDiff(arr1, arr2));
// Array empty.
console.log(getArrayDiff([], arr2));
console.log(getArrayDiff(arr1, []));
console.log(getArrayDiff([], []));

// Wrong parameter
console.log(getArrayDiff(arr1, e2));
console.log(getArrayDiff(e1, arr2));
console.log(getArrayDiff(e1, e2));
console.log('String assigned: ', getArrayDiff(arr1, str1));
console.log('String assigned: ', getArrayDiff(str2, arr1));
console.log(getArrayDiff(not, arr1));
console.log('Float assigned: ', getArrayDiff(flt, arr1));

// ----- TESTING - END ----------------------

// The actual function
// ------------------------------------------
// Returns the numbers which are contained in
//  ONE of two arrays.

// 1. Parameter: Array with ONLY integer values.
// 2. Parameter: Array with ONLY integer values.

// Return: Array with the difference set.
//  ! Returns null in case of error.

function getArrayDiff(arr1, arr2) {
  var ret = [];
  var merged = [];
  var current;
  var i;
  var sum;
  var isArray = Array.isArray;

  if (!isArray(arr1) || !isArray(arr2))
    return null;

  if (!arr1.length && arr2.length) {
    return arr2;
  } else if (arr1.length && !arr2.length) {
    return arr1;
  } else if (!arr1.length && !arr2.length) {
    return [];
  }

  merged = arr1.concat(arr2);

  merged.sort(function(a, b) {
    return a - b;
  });

  for (i = 0; i < merged.length; i++) {
    sum = 0;

    if ( isNaN(merged[i]) ||
         typeof merged[i] !== 'number' ||
         merged[i] % 1 !== 0 )
      return null;

    if (merged[i] !== current || i === 0) {
      current = merged[i];
    } else {
      continue;
    }

    if (arr1.indexOf(current) !== -1)
      sum++;

    if (arr2.indexOf(current) !== -1)
      sum++;

    if (sum === 1)
      ret.push(current);
  };

  return ret
}

Feedback

Use JSDoc comment style for the function, this style is widely used and most developers understand it.
Add comments inside your function to describe what every variable is used for and what every section is doing.
Do all the validations before starting to compute the result. So instead of checking if the elements are numbers while constructing the result; you should check them before.
Whenever you find yourself doing something twice, extract it to a separate function.
Throwing an error when it happens is more useful then returning null, it will give the user of your code a hint about what is going wrong and stops the execution of the code instead of using null as an array or checking the result every time your function is called.

Refactored code

/**
 * Checks if the given argument is an integer or not.
 * @param  {Mixed}   n the argument to check
 * @return {Boolean}   TRUE if n is an integer; FALSE otherwise.
 */
function isInteger (n) {
    return !isNaN(n) && typeof n === 'number' && n % 1 === 0;
}

/**
 * Checks if the given argument is an array of integers.
 * @param  {Mixed}  array the argument to check.
 * @return {Boolean} TRUE if the given array is an array of integers
 *                   (or an empty array); FALSE otherwise.
 */
function isArrayOfIntegers (array) {
    return Array.isArray(array) && array.length == array.filter(isInteger).length;
}

/**
 * Gets an object in which the keys are the items of the given array
 * and the values are TRUE.
 * @param  {Array} array Array of simple elemets (numbers or strings)
 * @return {Object}
 */
function toIndex (array) {
    return array.reduce(function(index, value) {
        index[value] = true;
        return index;
    }, {});
}

/**
 * Gets the array of elemets which are present exclusively in one of the arrays.
 * @param  {Array} array1 First array of integers.
 * @param  {Array} array2 Second array of integers.
 * @return {Array} Array of the integers which are present
 *                 exclusively in one of the two arrays.
 * @throws {Error} If one of the two arguments is not an array of integers.
 */
function getArrayDiff (array1, array2) {
    // Validating the arrays
    if (!isArrayOfIntegers(array1) || !isArrayOfIntegers(array2)) {
        throw Error('One of the given parameters is not an array of integers');
    }

    // Making objects from the arrays. Objects in which the keys
    // are the values of the arrays which will allow to search
    // the elements in logarithmic time
    var index1 = toIndex(array1);
    var index2 = toIndex(array2);

    // Computing the union of the arrays
    var result = array1.concat(array2);

    // Removing elements which are present in the two arrays
    result = result.filter(function(item){
        return index1[item] === undefined || index2[item] === undefined;
    });

    return result;
}

Hope this will help you and waiting for other members to give their suggestions and improve this code.


Recreate Mario's half-pyramids using hashes (#) from CS50 I currently working on CS50. There is a problem where we need to recreate half-pyramids using hashes (#). The hashes are to substitute blocks like in Mario.
This is also the first time I'm using Python. I don't really know Python. So I just try to calculate myself based on my experience from C language.
from sys import exit
from cs50 import get_int

def main():
    while True:
        num = get_int("Height: ")
        if num >= 1 and num <= 8:
            break

    print_pyramid(num)
    exit(0)

def print_pyramid(n):
    for i in range(n):
        print_spaces(i, n)              # calling two function that print spaces and hash
        print_hash(i)                   # after print every spaces and hash in each row,
        print()                         # print new line for new row

def print_spaces(i, n):
    for j in range(n-(i+1)):            # those calculation means that we print space one less than the height      ......#  (seven spaces)
        print(" ", end="")              # if height = 8     8-(i+1)     'i' is increment from above function        .....##  (six spaces)

def print_hash(i):
    for l in range(i+1):                # print the hash by increasing it   (i+1)
        print("#", end="")              # i = 0  # / i = 1  ##

    l = 1                               # there's two spaces between two pyramid. at the print_spaces function range(n - i+1)
    n = 4                               # so initialize l = 1 and n = 4, would always give (4-(1+1)) = 2
    print_spaces(l, n)

    for l in range(i+1):                # print rigth hash 'pyramid'
        print("#", end="")

main()

I always love to work with complicated idea using a lot of function for their different purpose.
I also got a 98% score from CS50.
What do you think of my code? Since this is my first time using Python, are there any Pythonic ways to simplify my program? (especially using function like I did).
Height: 4
       #  #
      ##  ##
     ###  ###
    ####  ####

Your code is quite nice. Your use of functions seem quite reasonable. And your style is nice too. Good job!

Calling print is expensive. To improve performance we can build the strings without using the for loop. For example to build \$n-(i + 1)\$ spaces we can use:
>>> n = 12; i = 1
>>> " " * (n - (i + 1))
'          '

To build the pyramid you can delegate the creation of the spaces to Python by using str.ljust and str.rjust.
These functions allow an easy to use way to build the pyramid.
>>> 'abc'.ljust(10)
'abc       '
>>> 'abc'.rjust(10)
'       abc'

Using print_spaces(l, n) is harder to understand than print("  ").

Your use of comments are not great. Most of them can be understood by just reading the code.
I only found one of your comments to help.
Because it explains something that isn't particuarly clear in clear English.
# there's two spaces between two pyramid. at the print_spaces function range(n - i+1)
# so initialize l = 1 and n = 4, would always give (4-(1+1)) = 2

However this is a sign that your code is complicated.
Which if you fix will make your comment redundent.

If we put these together we can greatly simplify your code.
from sys import exit
from cs50 import get_int

def main():
    while True:
        num = get_int("Height: ")
        if num >= 1 and num <= 8:
            break

    print_pyramid(num)
    exit(0)

def print_pyramid(n):
    for i in range(n):
        hashes = "#" * (i + 1)
        print(hashes.rjust(n) + "  " + hashes)

main()

There are still some more improvements that can be made.

Using sys.exit(0) is abnormal. It is normally best to just let Python handle that for you.

We can add an if __name__ == "__main__" guard to prevent your code from running if you import it accidentally.
This is because if you use import filename you can import this program by accident and we don't want to run the main function if that happens.

I personally am a fan of always using the < operator.
It just feels natural to see numbers ordered from smallest to largest, left to right.

We can Pythonify the print in print_pyramid by using the Format Specification Mini-Language.
There are three parts to this.

By using a format string we can enter values where we want them in a formatted string.
>>> a = 1; b = 2; c = 3
>>> f"{b}:{a} -> {c}"
'2:1 -> 3'
>>> "{b}:{a} -> {c}"  # Not a format string
'{b}:{a} -> {c}'
>>> "{b}:{a} -> {c}".format(a=a, b=b, c=c)  # Old style format strings
'2:1 -> 3'

We can use the alignment option to align the values where we want them to.
>>> a = 1; b = 2; c = 3
>>> f"{b:>3}:{a:<3} -> {c:^3}"
'  2:1   ->  3 '

We can enter the width using the same {} syntax as the values.
>>> a = 1; b = 2; c = 3; width = 3
>>> f"{b:>{width}}:{a:<{width}} -> {c:^{width}}"
'  2:1   ->  3 '

from sys import exit
from cs50 import get_int

def main():
    while True:
        num = get_int("Height: ")
        if 1 <= num and num <= 8:
            break
    print_pyramid(num)

def print_pyramid(n):
    for i in range(n):
        hashes = "#" * (i + 1)
        print(f"{hashes:>{n}}  {hashes}")

if __name__ == "__main__":
    main()


Project Euler #2 (classic) - Sum of even fibonacci numbers below 4 million I'm looking to use LISP as best I can, not just get the right answer. This is very early on in my LISP career so feedback is welcome and exciting! Recently asked about Project Euler #1, got some feedback and incorporated into this answer for #2, but I know there's a LOT left to be learned!

Description of Problem #2

  Each new term in the Fibonacci sequence is generated by adding the
  previous two terms. By starting with 1 and 2, the first 10 terms will
  be:

  1, 2, 3, 5, 8, 13, 21, 34, 55, 89, ...

  By considering the terms in the Fibonacci sequence whose values do not
  exceed four million, find the sum of the even-valued terms.

(defun sum (L)
  "sum a list"
  (reduce #'+ L))

(defun filter (L predicate)
  "filters a list"
  (remove-if (complement predicate) L))

(defun fiboNums (maxNum)
  "list of fibonacci numbers that are below maxNum"
  (setq A 1)
  (setq B 2)
  (setq next (+ A B))
  (loop while (< B maxNum) do
    (setq next (+ A B))
    (setq A B)
    (setq B next)
    collecting A))

(defvar answer (sum (filter (fiboNums (* 4 1000 1000)) #'evenp)))
(print answer)

Notation

It is a common convention in Lisp to avoid uppercase (or mixed lowercase and uppercase) letters in identifiers (symbols), so for instance use l instead of L, or fibo-nums instead of fiboNums, etc.

Variables

If you type your function in a Common Lisp interpreter/compiler, you will receive warnings undefined variable for A B and next. This is because you should introduce local variables before using them with the let or let* special operators (let* is needed if you need to initialize variables with the value other variables introduced). Then, you can assign these variables, as well as all the other type of variables, with setf, that can assign several variables in sequence.  So, for instance:

(defun fibo-nums (max-num)
  "list of fibonacci numbers that are below max-num"
  (let* ((a 1)
         (b 2)
         (next (+ a b)))
  (loop while (< b max-num)
    do (setf next (+ a b)
             a b
             b next)
    collecting a)))

Primitive operators

The definition of filter is not necessary, your definition is equivalent to the primitive function remove-if-not.

Algorithm

In this case there is no need of generate a list of Fibonacci numbers and then visiting it, it is sufficient to sum the numbers while generating them:

(defun sum-of-even-fibo (max-num)
  (loop for a = 1 then b
     for b = 1 then next
     for next = (+ a b)
     while (< a max-num)
     when (evenp a)
     sum a))

Finally, since loop is a powerful macro with a complex syntax, here is an alternative version with the more simple macro do*, which has also a syntax more “lisp-style”:

(defun sum-of-even-fibo (max-num)
  (do* ((sum 0)
        (a 1 b)
        (b 1 next)
        (next (+ a b) (+ a b)))
       ((> a max-num) sum)
    (when (evenp a)
      (incf sum a))))


Verifying password strength using JavaScript I have a function which verifies the "Password" field and suggests the user to enter a strong password.  I also have a label named "Password Strength", referring to the strength of a password (very weak, weak, medium, etc).

I'm just wondering if there is a better way to re-write this code.

function chkPasswordStrength(txtpass,strenghtMsg,errorMsg)
   {
     var desc = new Array();
     desc[0] = "Very Weak";
     desc[1] = "Weak";
     desc[2] = "Better";
     desc[3] = "Medium";
     desc[4] = "Strong";
     desc[5] = "Strongest";

     errorMsg.innerHTML = ''
     var score   = 0;

     //if txtpass bigger than 6 give 1 point
     if (txtpass.length > 6) score++;

     //if txtpass has both lower and uppercase characters give 1 point
     if ( ( txtpass.match(/[a-z]/) ) && ( txtpass.match(/[A-Z]/) ) ) score++;

     //if txtpass has at least one number give 1 point
     if (txtpass.match(/\d+/)) score++;

     //if txtpass has at least one special caracther give 1 point
     if ( txtpass.match(/.[!,@,#,$,%,^,&,*,?,_,~,-,(,)]/) ) score++;

     //if txtpass bigger than 12 give another 1 point
     if (txtpass.length > 12) score++;

     strenghtMsg.innerHTML = desc[score];
     strenghtMsg.className = "strength" + score;

  if (txtpass.length < 6)
  {
  errorMsg.innerHTML = "Password Should be Minimum 6 Characters"
  errorMsg.className = "errorclass"
  }

   }

<%= label "Password",:mandatory=>true %>

<input id="user_password" type="password" size="30" name="user[password]"  onkeyup="chkPasswordStrength(this.value,document.getElementById('strendth'),document.getElementById('error'))">

Password Strength

Dislcaimer: I'm not a security researcher and the following answer is compiled from my own, humble knowledge. The math is very basic and there are many things to consider, if in doubt, pay Security a visit. Also there are many factors that can kill password security completely, for example the user themselves or social engineering.

In this case I'm not reviewing the code, but I'm reviewing your program logic.

A short talk about password security

Let's first define the rules you use (the term "special chars" here for further on refers to the set !@#$%^&*?_~-()):

Password length > 6: 1 Point
Password length > 12: 1 Point
Password contains at least one lower and one upper case letter: 1 Point
Password contains at least one digit: 1 Point
Password contains at least one of the special chars: 1 Point

Your scale goes from 0 (Very Weak) to 5 (Strongest). That means a password based on length can never go beyond 2 (Better), but a short password can be 4 (Strong). This is bad if we keep in mind that strong passwords do not necessarily use a broad character set.

The following passwords are considered Strong by your algorithm:

abcdE$1
qwert!1
1111Aa@

Now these do not look like strong ones to me, let's have a look at how many possible combinations such a password has. The total character set for these passwords is 26 (lower) + 26 (upper) + 10 (digits) + 14 (special) = 76.

  76^7 = ~1.4 * 10^13 = ~14 trillion

So an attacker which knows the character set and the length of the password, has to search roughly 14 trillion combinations until they find the password. Though, that is the worst case scenario, could be that they find it on the third try, but could be that it is the next to the last. Todays computers can do more then 500 million guesses per second, that's roughly 8 hours...this can not be considered secure in any way.

Let's have a look at the other side of the spectrum, the following password is only considered Better:

  thisisaverylongpasswordbecause (This is a very long password because)

It only has a character set of 26 and a length of 30 (the maximum for your system if I've seen this correctly).

  26^30 = ~ 2.8 * 10^42 = ~2.8 tredecillion

I have no idea what that number is supposed to mean, so let's compare it. Okay, that doesn't help either...maybe if we put it into a timeframe, 500 million guesses per second again:

  ~1.8 * 10^26 years

That's actually good news! That's well before the heat death of the universe.

So, less nonsense, more talk: What's going on here?

The strength of a password can not reliable be defined. But we know two things:

The longer it is, the more combinations you have to go through.
The more different characters it uses, the more combinations you have to go through.

The difference is that one adds to the base, and the other to the exponent. A higher exponent weighs in heavier than a higher base and yields more total combinations. So whenever possible use a passphrase instead of a password.

Despite being, seemingly, vulnerable to dictionary attacks, the Oxford Dicitonary holds 300 thousand main entries, so if we have a passphrase of 4 words, that's:

  300000^4 = 8.1 * 10^21

And that does not take into account possible misspellings, if the words start uppercase, lowercase or mixed and different languages.

Make the users favor passphrases and long, easier to remember passwords then complicated short ones.

Sell it as a feature!

This will make your marketing department happy, you just implemented support for the more secure passphrases into your software! Let your users know about it by simply adding a short information to the password box:

  We support passphrases up to a length of XX characters!

  Passphrases are more secure and easier to remember than ordinary passwords.

And some short information about how to "create" one, and drop any indicator if the password is secure or not.

Who you can and can't help

There are quite many user groups out there, the question is who you want to reach with this help. Let's define three groups:

The "average user", which does not understand why they need to press a button labeled "Delete" to delete something.
The normal user, always ready to learn and looks beyond their own nose.
The technical user, already uses a password manager and/or passphrases.

No matter how hard you try, you will not be able to help the "average user". If you tell them to use a password of minimum length 6, they will use "123456" or "asdfgh". If you tell them to use a password of minimum length 12, they will use "asdfghjkzxcvbn" or "000000000001". They are beyond hope and will actively workaround security measures you implement. Just let 'em be.

The "normal user" on the other hand is ready to learn something new, telling them about passphrases and that you support them will make them want to use passphrases. If your small help text and the rest of the system in that moment is helpful, they will use a passphrase which is not something absolutely stupid.

The "technical user" will be pleased that you support a XX length password and passphrases. They most likely already use a strong and random password and a password manager or passphrases, but that you don't limit them in any way will make them happy.

Secure passwords stored insecure are insecure

It doesn't matter how good the password or passphrase is, if you fail to store in a secure manner, they are insecure.

We will completely leave out social engineering or "harmful" users (there are people after all who have their creditcard PIN on a note in the same wallet), we can't help them.

But what is important that if somebody manages to break into your server and manage to copy the user database, they must not have access to all accounts. This is achieved by simply hashing the passwords with an appropriate method. Even if the attacker now has all the usernames and e-mail addresses, the passwords are still safe.

It will also make the marketing department happy that, despite you've been hacked, they can tell the users that their accounts are still safe (password change advised, obviously).

The great news is, if you hash the passwords there's absolutely no reason to limit password length at all! They will all end up as the same length hash in your database, so you can let me use that sentence from that book as a passphrase.

Ask the right people the right questions

Last but not least, defining what is a strong password depends on many things. If in doubt, pay the people over at Security a visit, that's what they're there for.


Streaming floating point data in Python2 I am in the process of creating a brain-computer interface, and one task is to stream electroencephalograph (EEG) data into a Python script for real-time analysis. Included in this question are two Python scripts: (1) to generate and stream fake EEG data (32 random floating points for testing purposes), and (2) to receive the EEG data and convert it to a numpy array of the shape (n_channels, n_samples). The second script extracts a sliding window of the last X seconds of data.

Generate and stream synthetic EEG data

# file is named "stream_eeg_realtime.py"
import random, threading, time

nominal_srate = 500.  # sampling rate in Hz. Must be float.
sleep_time = 1/nominal_srate

# Define all of the channel names.
channels = ['Fp1','Fp2','AF3','AF4','F3','F4','F7','F8','FC5','FC6','T7','T8',
'FC1','FC2','C3','C4','CP5','CP6','P7','P8','CP1','CP2','P3','P4','01','02',
'PO3','PO4','Oz','Pz','Cz','Fz']

one_sample = [round(random.uniform(-4.,4.), 5) for _ in channels]
data = []

datalock = threading.RLock()
def synthesize_data():
    while True:
        with datalock:
            data.append(one_sample)
        time.sleep(sleep_time)

t1 = threading.Thread(target=synthesize_data, name="EEG")
t1.daemon = True
t1.start()
print("Now streaming synthetic data...")

Receive and convert data to Numpy array (Jupyter Notebook)

In [1]:
import numpy as np
import stream_eeg_realtime as ser

DATA_DURATION = 20
sfreq = float(ser.nominal_srate)
last_rows_index = int(DATA_DURATION*sfreq)
thread_lock = ser.datalock
def get_data(raw_data, index, thread_lock, full_data=False):
    # Data comes in as array-like with shape (n_samples, n_channels).
    with thread_lock:
        if full_data==False:
            d = [row[:] for row in raw_data[-index:]]
        # If we want all of the data, instead of the X-second window.
        elif full_data==True:
            d = [row[:] for row in raw_data]
    data = np.array(d, dtype="float64").T
    return data

In [2]:
raw_array = get_data(ser.data, last_rows_index, thread_lock, full_data=False)
print raw_array.shape

This code has to be as fast as possible. I optimized the code as much as I could as an advanced-beginner, but can I make it faster? Is it OK to use threading in this case, or would multiprocessing be better? (I have no idea how to use multiprocessing)

Here are some things I tried before arriving at the code I posted. At first, I tried to append "columns" to the nested lists with list comprehension. That was fast, but it turns out it is much faster to append "rows" (no list comprehension needed) and then transpose. I also tried to deepcopy the list in the second script, and then I used nested list comprehension to make a copy. Using the slicing syntax in the list comprehension was the fastest method to copy the list.

datalock seems to be unneeded, since data.append is thread-safe and iterating while appending is safe.

multiprocessing might be faster, but it might be slower. It depends on where you're bottlenecked. In particular, data transfer is slower with multiple processes.

The slow aspect here is primarily repeated conversions to numpy.ndarray objects. This can be avoided by only doing the translation once, which is easiest in the producer thread:

data.append(np.array(one_sample, dtype="float64"))

get_data is then simpler as

def get_data(raw_data, number_values=None):
    if number_values == 0:
        return []
    return np.array(raw_data[-number_values or 0:]).T

If the block size you want is bounded by a fixed limit, it's probably better to write into a circular numpy buffer of the right shape. This way, you can just do a single concatenation, rather than a concatenation of a large number of arrays.


Hover on bootstrap card overlay Just created an overlay on hover following a Youtube tutorial - but I adjusted some css to make it responsive.
I'd like to hear hear some reviews or suggestions where I can improve.

#Projects .card {
  position: relative;
  overflow: hidden;
}

#Projects .card:before {
  position: absolute;
  content: '';
  width: 80%;
  height: 220%;
  background:rgba(199, 0, 57, .9);
  top: -50%;
  left: -100%;
  z-index: 1;
  transform: rotate(20deg);
  transform-origin: center top 0;
  transition: .5s;
}

#Projects .card:hover:before {
  left: 10%;
}

#Projects .card .card-text {
  width: 100%;
  padding: 0 20px;
  position: absolute;
  top: -150%;
  color: #fff;
  left: 0;
  z-index: 2;
  transition: .5s;
}

#Projects .card:hover .card-text {
   top: 20%;
}

@media (max-width: 992px) {
  #Projects .card:hover .card-text {
    top: 15%;
 }

  #Projects  .card-text a {
    font-size: 11px;
  }

  #Projects .btn {
    padding: 5px 15px;
  }

  #Projects  .card-text h3 {
    font-size: 18px;
  }

  #Projects  .card-text h5 {
    font-size: 18px;
  }

}

@media (max-width: 575px) {

  #Projects .btn {
    padding: 5px 24px;
    font-size: 1.1rem;
  }

  #Projects  .card-text h3 {
    font-size: 1.5rem;
  }

  #Projects  .card-text h5 {
    font-size: 1.5rem;
  }

}
 <link rel="stylesheet" href="https://stackpath.bootstrapcdn.com/bootstrap/4.5.2/css/bootstrap.min.css"
    integrity="sha384-JcKb8q3iqJ61gNV9KGb8thSsNjpSL0n8PARn9HuZOnIxN0hoP+VmmDGMN5t9UJ0Z" crossorigin="anonymous">

Project 1

Some text...

            View

Project 2

Some text...

            View

In large projects is important to save as much space as possible.
Your selectors are way to specific for example :
#Projects .card .card-text {}

Whereas this will do:
.card-text {}

If the purpose of this was a different card-text in a different card you can add another class to card-text as a modifier


Reading from an Excel sheet I have created a class for reading data from an Excel sheet and holding on the variable.  I am not sure if my code is optimized, meaning I will be reading heavily from Excel.

Is there a room to further optimized this code?  I will be using this class heaving in my code, and I wanted to get some opinions/feedback.

I will be calling something like this:

ExcelReader xl = new ExcelReader();
string sss = xl.ExcelOpenSpreadsheets("mysheet");
//etc...

public class ExcelReader
{
    Application _excelApp;

    public ExcelReader()
    {
        _excelApp = new Application();
    }

    public string ExcelOpenSpreadsheets(string sheetName)
    {
        string _txt = string.Empty;
        try
        {
            Workbook workBook = _excelApp.Workbooks.Open("filename_here",....);
            _txt = ExcelScanIntenal(workBook, sheetName);
        }

        catch
        {
            //
            // Deal with exceptions.
            //
        }
        return _txt;
    }

    private string ExcelScanIntenal(Workbook workBookIn, string sheetName)
    {
        Worksheet sheet = workBookIn.Sheets[sheetName] as Worksheet;

        Range a1 = sheet.get_Range("A1", "B2");
        if (a1 != null)
        {
            string formattedText = r.Text;
        }

        return formattedText;
    }

I'd change the code so that every implicit constant value is an argument in the constructor, with default values.

In addition, I guess this excel application is the COM API for Excel, so perhaps a better approach would be using the IDisposable interface and implementation.

I changed the typo and now the inner method's name is ExcelScanInternal (and not ExcelScanIntenal).

So the result looks like this (I didn't compile it, so it might have syntax errors):

public class MyRange
{
    public string @From { get; set; }
    public string @To { get; set; }
}

public class ExcelReader : IDisposable
{
    Application _excelApp;
    string _fileName;
    MyRange _scanRange;

    public ExcelReader() : this ("the_file_name", new MyRange{From = "A1", To = "B2"})
    {
    }

    public ExcelReader(string fileName, MyRange scanRange)
    {
        if (null == fileName) throw new ArgumentNullException("fileName");
        _fileName = fileName;
        if (null == scanRange) throw new ArgumentNullException("scanRange");
        _scanRange = scanRange;
        _excelApp = new Application();
    }

    public string ExcelOpenSpreadsheets(string sheetName)
    {
        string _txt = string.Empty;
        try
        {
            Workbook workBook = _excelApp.Workbooks.Open(_fileName,....);
            _txt = ExcelScanInternal(workBook, sheetName);
        }

        catch
        {
            //
            // Deal with exceptions.
            //
        }
        return _txt;
    }

    private string ExcelScanInternal(Workbook workBookIn, string sheetName)
    {
        Worksheet sheet = workBookIn.Sheets[sheetName] as Worksheet;

        Range a1 = sheet.get_Range(_scanRange.From, _scanRange.To);
        if (a1 != null)
        {
            string formattedText = a1.Text;
        }

        return formattedText;
    }

    public void Dispose()
    {
        _excelApp.Quit();
    }
}


Quick Sum TopCoder (Brute Force solution) Given a string of digits, find the minimum number of additions
  required for the string to equal some target number. Each addition is
  the equivalent of inserting a plus sign somewhere into the string of
  digits. After all plus signs are inserted, evaluate the sum as usual.
  For example, consider the string "12" (quotes for clarity). With zero
  additions, we can achieve the number 12. If we insert one plus sign
  into the string, we get "1+2", which evaluates to 3. So, in that case,
  given "12", a minimum of 1 addition is required to get the number 3.
  As another example, consider "303" and a target sum of 6. The best
  strategy is not "3+0+3", but "3+03". You can do this because leading
  zeros do not change the result.

  Write a class QuickSums that contains the method minSums, which takes
  a String numbers and an int sum. The method should calculate and
  return the minimum number of additions required to create an
  expression from numbers that evaluates to sum. If this is impossible,
  return -1.

  TopCoder link

So I came up with a brute force solution of checking all possible solutions to find the min. Is there a better way to do this?

public class QuickSums {

  public static void main(String args[]) {
    String input = "99999";
    int total = 45;
    int res = minSums(input, total);
    System.out.println(res);
  }

  // With at most 10 digits(constraint given), there are at most 2^9 ways to insert plus signs into the string.
  // Therefore, there are at most 2^9 possibilities to consider. We can use recursion to go through
  // all possibilities and keep track of the minimum number of additions required
  public static int minSums(String numbers, int sum) {
    int N = numbers.length();

    // base cases
    if (N <= 3 && sum == Integer.parseInt(numbers)) {
      return 0;
    }

    else if (N == 2 && Integer.parseInt(numbers.substring(0, N - 1))
        + Integer.parseInt(numbers.substring(N - 1)) == sum) {
      return 1;
    } else if (N == 2 && Integer.parseInt(numbers.substring(0, N - 1))
        + Integer.parseInt(numbers.substring(N - 1)) != sum) {
      return -1;
    }
    else if (N == 1 && Integer.parseInt(numbers) != sum) {
      return -1;
    }

    // solution
    else {

      int lenOFStr = N-1;
      int possibleCombination = (int) Math.pow(2, lenOFStr);

      // numbers will contain between 1 and 10 characters, inclusive.
      int min = Integer.MAX_VALUE;
      StringBuilder tempString;
      for (int i = 0; i < possibleCombination; i++) {
        String plus = Integer.toBinaryString(i);
        while (plus.length() < lenOFStr) {
          plus = "0" + plus;
        }
        // System.out.println(plus);

        // Add plus sign to the string
        tempString = new StringBuilder(numbers);
        int len = N;
        for (int k = 0; k < lenOFStr ; k++) {
          if (plus.charAt(k) == '1') {
            len = len + 1;
            int offset = len-N+k;
           // System.out.println(offset);
            tempString.insert(offset, " ");
          }
        }
        // System.out.println(tempString);

        // compute the sum
        String[] arr = tempString.toString().split(" ");
        long tempSum = 0;
        for (String s : arr) {
          tempSum +=Long.parseLong(s);
        }

        // check if the sum is same
        if (tempSum == sum) {
          min = Math.min(arr.length-1, min);
        }

      }
      return (min == Integer.MAX_VALUE) ? -1 : min;

    }
  }

}

This program works fine for all inputs but I'm wondering, there must be a dynamic programming solution to this? Or something better?

public static int minSums(String numbers, int sum) {
  int N = numbers.length();

variables should be named using camelCase casing so n would be better, but what does N/n represent ? Its the number of digits contained in the String numbers which isn't named correctly because there is only one number passed which consists of multiple digits. So a better way would be like so

public static int minSums(String number, int sum) {
  int digits = number.length();

but as the default indentation is 4 spaces this should be like so

public static int minSums(String number, int sum) {
    int digits = number.length();

This if..else if..else construct can be simplified because for the first cases you are returning a value the else if..else aren't needed and will save some horizontal spacing if removed. You should have the two cases where you check for N == 2 into one single if with a nested if..else like so

// changed N to digits

if (digits <= 3 && sum == Integer.parseInt(numbers)) {
    return 0;
}

if (digits == 2) {
    if (Integer.parseInt(numbers.substring(0, N - 1))
       + Integer.parseInt(numbers.substring(N - 1)) == sum) {
        return 1;
    } else {
        return -1;
    }
}

if (digits == 1 && Integer.parseInt(numbers) != sum) {
  return -1;
}

// solution

Commented out code is dead code and should be removed because it is only adding noise to the code which decreases readability.

Let your variables and operators have some space to breathe. Something like int offset = len-N+k; isn't just as readable as int offset = len -N + k;.


Consistent way to handle transient timeouts with WCF calls (timeouts, unreliable network, server load, etc) I'm currently using the following code to manage calls to WCF services that are unreliable, and or suffer performance load issues due to contention.

Right now this code is limited to solving issue with types that return void. It exponentially backs off from a subsequent call from the server, preventing unhelpful "hammering" of the server.

Question

Is there any possible way I can abstract this into something more generic for varying return types?
Have I caught all the appropriate exceptions worthy of a retry?
What is the most appropriate way I can communicate to the calling process what is going on with this code, perhaps offering an opportunity to cancel early?
Any other enhancements you may think of...

Usage Example:

Service<IOrderService>.Use(orderService=>
{
  orderService.PlaceOrder(request);
}

Source code

public delegate void UseServiceDelegate<T>(T proxy);

public static class Service<T>
{
    public static ChannelFactory<T> _channelFactory = new ChannelFactory<T>("");

    public static void Use(UseServiceDelegate<T> codeBlock)
    {
        IClientChannel proxy = (IClientChannel)_channelFactory.CreateChannel();
        bool success = false;

       Exception mostRecentEx = null;
       int millsecondsToSleep = 1000;

       for(int i=0; i<5; i++)  // Attempt a maximum of 5 times
       {
           try
           {
               codeBlock((T)proxy);
               proxy.Close();
               success = true;
               break;
           }

           // The following is typically thrown on the client when a channel is terminated due to the server closing the connection.
           catch (ChannelTerminatedException cte)
           {
              mostRecentEx = cte;
               proxy.Abort();
               //  delay (backoff) and retry
               Thread.Sleep(millsecondsToSleep  * (i + 1));
           }

           // The following is thrown when a remote endpoint could not be found or reached.  The endpoint may not be found or
           // reachable because the remote endpoint is down, the remote endpoint is unreachable, or because the remote network is unreachable.
           catch (EndpointNotFoundException enfe)
           {
              mostRecentEx = enfe;
               proxy.Abort();
               //  delay (backoff) and retry
               Thread.Sleep(millsecondsToSleep * (i + 1));
           }

           // The following exception that is thrown when a server is too busy to accept a message.
           catch (ServerTooBusyException stbe)
           {
              mostRecentEx = stbe;
               proxy.Abort();

               //  delay (backoff) and retry
               Thread.Sleep(millsecondsToSleep * (i + 1));
           }
           catch (TimeoutException timeoutEx)
           {
               mostRecentEx = timeoutEx;
               proxy.Abort();

               //  delay (backoff) and retry
               Thread.Sleep(millsecondsToSleep * (i + 1));
           }
           catch (CommunicationException comException)
           {
               mostRecentEx = comException;
               proxy.Abort();

               //  delay (backoff) and retry
               Thread.Sleep(millsecondsToSleep * (i + 1));
           }
           catch(Exception )
           {
                // rethrow any other exception not defined here
                // You may want to define a custom Exception class to pass information such as failure count, and failure type
                proxy.Abort();
                throw ;
           }
       }
       if (success == false && mostRecentEx != null)
       {
           proxy.Abort();
           throw new Exception("WCF call failed after 5 retries.", mostRecentEx );
       }

    }
}

One clean up is to consolidate the common code in the catch blocks, by putting it into a function. Other ideas:

5 tries is not many for WCF in general. I would increase it to 50 or so. If you do this, you need a ceiling to multiple applied to to the minimum sleep time.
The milliseconds to sleep should be initialized to a semi-random value: a minimum value, plus a random fraction of an additional amount. Usually I make the additional amount equal to the minimum value. This will keep all your clients from hammering the server with the same rhythm, in the event all clients start at the same time.

With regards to questions:

Make the method generic, with a type parameter defining the return value.
I usually have only two catch blocks, one for FaultException to communicate exceptions explicitly from server to client, and one for Exception, to catch everything else. The Exception block is where you initiate a retry. Read about FaultContractAttribute.
The only good way to allow cancellation is to run all of this in a background thread, possibly using BackgroundWorker.


Scala code to insert a row into Hbase Below is a case class and a companion object with a method putRecord which adds a new row into Hbase using Put API. This code has some repetitive parts. Is there a way to abstract away the repetitive code

case class HbaseRow(_id: Option[String], field1: Option[Array[String]], field2: Option[Array[String]], field3: Option[String], field4: Option[Long])

object HbaseRow {
    def putRecord(row: HbaseRow): Put = {
        val put = new Put(Bytes.toBytes(row._id.get))

        def addColumn[T](columnName: String,value: T): Put = {
          val columnFamily:Array[Byte] = Bytes.toBytes("cf")
          val col: Array[Byte] = Bytes.toBytes(columnName)
          // Repetative, almost same method call but for different types
          value match {
            case v: String => put.addColumn(columnFamily, col,Bytes.toBytes(v))
            case v: Long => put.addColumn(columnFamily, col,Bytes.toBytes(v))
            case v: Boolean => put.addColumn(columnFamily, col,Bytes.toBytes(v))
            case v: Array[String] => put.addColumn(columnFamily, col,Bytes.toBytes(v.mkString(",")))
          }
        }

        //Repetitive will be tedious as columns increase.
        row.field1.map(value => addColumn[Array[String]]("f1",value.mkString(",")))
        row.field2.map(value => addColumn[Array[String]]("f2",value.mkString(",")))
        row.field3.map(value => addColumn[String]("f3",value))
        row.field4.map(value => addColumn[Long]("f4",value))

        put
    }
}

In def addColumn you can extract bytes and then add them to put:
val bytes = value match {
  case v: String => Bytes.toBytes(v)
  case v: Long => Bytes.toBytes(v)
  case v: Boolean => Bytes.toBytes(v)
  case v: Array[String] => Bytes.toBytes(v.mkString(","))
}
put.addColumn(columnFamily, col, bytes)

I'm not sure that Scala can check generic types in pattern matching, so Array[String] may be checked only for class Array. If it is than the compiler will warn you.
2nd, if you don't use result, you can use foreach instead of map.
3rd value.mkString(",") is String so value type is also String, not Array[String]. Of course you can simple declare def addColumn(columnName: String, value: Any), or first create sealed trait ColumnValue and use def addColumn(columnName: String, value: ColumnValue).
According to 3rd, you can use:
List("f1" -> row.field1, ..., "f4" -> row.field4).foreach {
  // you can use only 2nd case if you handle `Array` in `def addColumn`
  case (columnName, field: Array[_]) => addColumn(columnName, field.mkString(","))
  case (columnName, field) => addColumn(columnName, field)
}


Create a SelfJoin with has_many I am trying to do a self-join.  I generated this migration to make a
correct primary.  This code works, but I'm not sure if this is the best way to do things.

class CreateFeedstocks < ActiveRecord::Migration[5.0]
  def change
    create_table :feedstocks do |t|
      t.string :code
      t.float :measure_one
      t.float :measure_two
      t.float :measure_three
      t.string :description
      t.integer :primary_feedstock_id, index: true

      t.timestamps
    end
    add_foreign_key :feedstocks, :feedstocks,
                     column: :primary_feedstock_id,
                     primary_key: :id
  end
end

One feedstock has one primary_feedstock and one primary_feedstock has
many feedstocks.

Does the format provided in the Rails Guides not work for you?

# Model ------------------------------------------------------------------------
class Feedstock < ApplicationRecord
  has_many :feedstocks,   foreign_key: "primary_feedstock_id"

  belongs_to :primary_feedstock, class_name: "Feedstock"
end

# Migration --------------------------------------------------------------------
class CreateFeedstocks < ActiveRecord::Migration[5.0]
  def change
    create_table :feedstocks do |t|
      t.references :primary_feedstock, index: true
      t.timestamps
    end
  end
end

EDIT: So I'm still not clear on the difference between add_foreign_key and t.references. According to this comment on SO, t.references will not create a foreign key constraint, but that was written 5 years ago, and the part of the Guides that the commenter mentioned is no longer there.

The API docs are abundantly clear about add_foreign_key creating a foreign key constraint at the database level; unfortunately, they're not so clear about t.references.

In any case, you will have to decide for yourself how to design your application. I've heard some people say that depending on ActiveModel is ‘the Rails Way’, but I haven't seen that position stated anywhere in the official docs. Alternately, others will tell you that implementing these checks at the database level is more performant and reliable, and I can't argue with that. I think in the end, the choice comes down to how strictly you believe in Rails' convention-over-configuration philosophy, and whether you're willing to sacrifice a little abstraction for performance and reliability.

EDIT 2: It appears that add_foreign_key alone is not sufficient to create a table column. You'll need t.references either way, followed by either the foreign_key: true option or a separate add_foreign_key directive.


Simple GUI animation in Java I wrote a simple animation of a ball moving across the screen. I was wondering if I used the static variable correctly. If not, how could I make it non-static so I could instantiate it in the main method whilst being able to access its getter methods in the listener inner class? Also, did I use the SwingUtilities.invokeLater() method properly?

Please let me know of any improvements that I can make.

import java.awt.*;
import javax.swing.*;

public class SimpleAnimation {
    private JFrame frame;
    private int x = 5;
    private int y = 5;

    static SimpleAnimation animation;

    public static void main(String[] args){
        animation = new SimpleAnimation();
        animation.buildGUI();

        // Use i to change co-ordinates x and y to produce a moving ball on the screen.
        for(int i = 0; i <= 300; i+=1){
            animation.setCoordinates(i, i);
            animation.frame.repaint();      // JVM uses new co-ordinates in paintComponent() to repaint the ball
            try{
                Thread.sleep(10);
            }catch(InterruptedException interruptedException){
                interruptedException.printStackTrace();
            }
        }
    }

    public void buildGUI(){
        frame = new JFrame("Simple animation");
        SwingUtilities.invokeLater(
                new Runnable() {
                    @Override
                    public void run() {
                        frame.setContentPane(new DrawPanel(new BorderLayout()));
                        frame.setSize(400, 500);
                        frame.setDefaultCloseOperation(WindowConstants.EXIT_ON_CLOSE);
                        frame.setLocationRelativeTo(null);
                        frame.setVisible(true);
                    }
                }
        );

    }

    // GETTERS
    int getX(){
        return x;
    }

    int getY(){
        return y;
    }

    // SETTERS
    void setCoordinates(int newX, int newY){
        x = newX;
        y = newY;
    }

    class DrawPanel extends JPanel{
        public DrawPanel(LayoutManager layout){
            super(layout);
        }

        @Override
        public void paintComponent(Graphics g){
            g.setColor(Color.ORANGE);
            g.fillOval(animation.getX(), animation.getY(), 100, 100);
        }
    }
}

I was wondering if I used the static variable correctly? If not, how could I make it non-static so I could instantiate it in the main method whilst being able to access its getter methods in the listener inner class?

I would prefer to not use a static variable for this, which you could easily achieve by passing the animation object to the panel:

public class AnimationPanel extends JPanel {

    private SimpleAnimation animation;

    public AnimationPanel(LayoutManager layout, SimpleAnimation animation) {
        super(layout);
        this.animation = animation;
    }

    @Override
    public void paintComponent(Graphics g) {
        super.paintComponent(g);
        g.setColor(Color.ORANGE);
        g.fillOval(animation.getX(), animation.getY(), 100, 100);
    }
}

With a simple animation class (in a real project, setcoordinates would more likely be updateCoordinates, as the animation model should contain its logic itself, ie it should know what to animate, instead of having the main class/controller manage this. the main class would just be a loop constantly calling updateCoordinates on the model, and repaint on the view):

public class SimpleAnimation {
    private int x = 5;
    private int y = 5;

    int getX(){
        return x;
    }

    int getY(){
        return y;
    }

    void setCoordinates(int newX, int newY){
        x = newX;
        y = newY;
    }
}

And then use it in your main like this:

public class AnimationMain {

    public static void main(String[] args) {
        SimpleAnimation animation = new SimpleAnimation();
        AnimationPanel animationPanel = new AnimationPanel(new BorderLayout(), animation);

        buildGUI(animation, animationPanel);

        for (int i = 0; i <= 300; i += 1) {
            animation.setCoordinates(i, i);
            animationPanel.repaint();
            try {
                Thread.sleep(10);
            } catch (InterruptedException interruptedException) {
                interruptedException.printStackTrace();
            }
        }
    }

    public static void buildGUI(SimpleAnimation animation, AnimationPanel animationPanel) {
        JFrame frame = new JFrame("Simple animation");
        SwingUtilities.invokeLater(
                new Runnable() {
                    @Override
                    public void run() {
                        frame.setContentPane(animationPanel);
                        frame.setSize(400, 500);
                        frame.setDefaultCloseOperation(WindowConstants.EXIT_ON_CLOSE);
                        frame.setLocationRelativeTo(null);
                        frame.setVisible(true);
                    }
                }
        );
    }
}

The main reason I would prefer this approach is that it follows the MVC approach (separating the code by model (SimpleAnimation), view (AnimationPanel), and controller (AnimationMain)). This seems a lot more flexible (with your approach, I even have problems extracting the main method to its own class, for example).

Misc

don't import *, instead make all your imports explicit, so readers know what classes you use.
your methods should be explicitly declared as private or public.
your spacing is sometimes off (eg i+=1; it should be i += 1, or the more customary i++).


Program to count vowels After reading a certain question on the Stack Overflow website, I tried to write a solution to the problem just for fun. I'm, however, left with the nagging feeling there is a beautiful one-liner that can be used instead.
The question's premises:

Create a function that will receive a string. The function will count each vowel in the string, and return a count of all the vowels, whether found or not, each in a tuple pair. Each tuple parir will be stored in a list.

Example:
>>> vowels_finder("This has some vowels")
[('a', 1), ('o', 2), ('u', 0), ('e', 2), ('i', 1)] # tuple pair of each vowel.
>>>

My attempt:
def vowels_finder(s):
    vowels = {'a':0, 'e':0, 'i':0, 'o':0, 'u':0}
    for el in s:
        if el in {'a', 'e', 'i', 'o', 'u'}:
            vowels[el]+=1
    vowels = [(key, pair) for key, pair in vowels.items()]
    return vowels

My code above is not commented, but I'm confident that its brevity will allow me to pass on this.
Questions:

Is there _any way, besides using a python library, that this can be condensed into a one-liner?
Would there be a way to not have to convert the vowels key's back into tuple pairs, and just have them be tuples in the beginning. eg: vowels = [('a', 0), ('e', 0), ('i', 0), ('o', 0), ('u', 0)] ?

language: Python 3.4

With Python it can be tempting to want to write one-liners; but, short code does not necessarily make for better code, and I'd like to review your code in a way that I think would make it more maintainable, flexible and professional, rather than shorter. (maybe someone else will address the one-liner request)
Type hints
Since you are using Python 3.x you could take advantage of the new type hints. According to PEP 484:

This PEP aims to provide a standard syntax for type annotations, opening up Python code to easier static analysis and refactoring, potential runtime type checking, and (perhaps, in some contexts) code generation utilizing type information.
Of these goals, static analysis is the most important. This includes support for off-line type checkers such as mypy, as well as providing a standard notation that can be used by IDEs for code completion and refactoring.

Even if you don't use static code analysis at the moment, type hints still have the advantage of making the code easier to read and understand.
In your case:
def vowels_finder(s: str) -> list:
    # ...

Reusable functions
The first thought I had looking at your function is that some logic could be extracted for more general reuse. For instance, this function could come in handy for other things:
def is_vowel(ch: chr, include_y: bool=False) -> bool:
    if include_y:
        return ch in ('a', 'e', 'i', 'o', 'u', 'y')
    else:
        return ch in ('a', 'e', 'i', 'o', 'u')

You will notice I also added support for optionally including "Y" as a vowel, which can be useful for certain contexts.
Note that I also used a tuple instead of set, since vowels won't change anyways and tuples are generally faster since they are immutable, and we don't need set operations in this case other than in membership, which tuples support as well.
Now we can simply do this in your vowels_finder function:
for el in s:
    if is_vowel(el):
        vowels[el]+=1

Main function improvements
I would expect a function named vowels_finder to do just that: look for a vowel, and return True if it finds one. Furthermore, I would expect a name like this to be an object/class "Thing", rather than a function which is usually named like "do something".
Let's call it count_individual_vowels instead. Also, now that we have a function for vowels with added functionality for "Y", we can very easily add this option to this function. Note that I have changed some of the variable names a bit to make them more clear:
def count_individual_vowels(input_str: str, include_y: bool = False) -> list:
    vowel_counts = {'a':0, 'e':0, 'i':0, 'o':0, 'u':0}
    if include_y:
        vowel_counts['y'] = 0
    for el in input_str:
        if is_vowel(el, include_y):
            vowel_counts[el] += 1
    return [(key, pair) for key, pair in vowel_counts.items()]

Bug / overlooked problem
After refactoring this I noticed a problem, see for illustration:
string1 = "My phrase has some vowels, pretty cool don't you think?"
print(count_individual_vowels(string1))
print(count_individual_vowels(string1, True))
string2 = string1.upper()
print('UPPER CASE')
print(count_individual_vowels(string2))
print(count_individual_vowels(string2, True))

Results:

[('u', 1), ('o', 6), ('i', 1), ('e', 4), ('a', 2)]
[('e', 4), ('a', 2), ('u', 1), ('y', 3), ('o', 6), ('i', 1)]
UPPER CASE
[('u', 0), ('o', 0), ('i', 0), ('e', 0), ('a', 0)]
[('e', 0), ('a', 0), ('u', 0), ('y', 0), ('o', 0), ('i', 0)]

This could of course cause problems, but thankfully the fix is extremely simple, just use .lower() method on strings in the functions
Here in the new helper function...

return ch.lower() in ('a', 'e', 'i', 'o', 'u')

And also when adding it into the dict, so that uppercase vowels get grouped into the lowercase count:
for el in input_str:
    if is_vowel(el, include_y):
        # here:
        vowel_counts[el.lower()] += 1

OrderedDict to keep vowels in order
As you've noticed, using a regular dict to store the counts results in the output vowels returns the values in arbitrary order. You could make it return always in the same order by using from collections import OrderedDict and just replacing this:
vowel_counts = {'a':0, 'e':0, 'i':0, 'o':0, 'u':0}
if include_y:
    vowel_counts['y'] = 0

By this a bit more verbose, but much nicer output:
vowel_counts = OrderedDict()
for vow in 'a', 'e','i', 'o', 'u':
    vowel_counts[vow] = 0
if include_y:
    vowel_counts['y'] = 0

See:

[('a', 2), ('e', 4), ('i', 1), ('o', 6), ('u', 1)]
[('a', 2), ('e', 4), ('i', 1), ('o', 6), ('u', 1), ('y', 3)]

Finally, here is a working demo on repl.it with all the above suggestions applied.


Aggregation of a collection of object's nested array-properties Use Case

So our lead programmer loves to follow the NIH Anti-Pattern and consequently I'm not allowed to use Underscore.js and can only use Vanilla JS... and we have to support IE8.

The goal is to produce an aggregation of unique values from nested collections buried within various collections of objects.

Desired Interface

The idea is to call a function to create an aggregation function give the list of the nested properties.

Example

The following would flatten all options collection that are buried under each object within the properties property.

var allUniqueOptions = aggregateByProperty('properties', 'options');
var allUniqueDoodadsOptions = aggregateByProperty('doodad', 'properties', 'options');

allUniqueOptions(myCollection);
allUniqueDoodadsOptions(myDoodadCollection);

Current Code

The code below works, but I'd like any advice on improving it for:

performance
readability
any further generalization

I have a jsFiddle created.

/* Returns the index of an object within a collection */
function arrayObjectIndexOf(arr, obj) {
    var search = JSON.stringify(obj);
    for ( var i = 0, k = arr.length; i < k; i++ ){
        if (JSON.stringify(arr[i]) == search) {
            return i;
        }
    };
    return -1;
};

/* Concatentates objects that don't exist in the source collection */
function concatIfNotExists(source, additions) {
    var result = source;
    for ( var i = 0, k = additions.length; i < k; i++ ) {
        var addition = additions[i];
        if (arrayObjectIndexOf(result, addition) === -1) {
            result.push(addition);
        }
    };
    return result;
};

/* Returns a nested collection buried within an object */
function getInnerCollection(obj, props) {
    var innerCollection = obj;

    for (var j = 0, l = props.length; j < l; j++) {
        if (innerCollection.hasOwnProperty(props[j])) {
            innerCollection = innerCollection[props[j]];
        } else {
            throw new TypeError("Property doesn't exist!");
        }
    }

    if (!innerCollection instanceof Array) {
        throw new TypeError("Inner property is not a collection!");
    }
    return innerCollection;
};

/* Returns an aggregation of objects based on collection properties */
function aggregateByProperty() {
    var props = Array.prototype.splice.call(arguments, 0);

    return function(objectCollection) {
        var result = [];
        for (var i = 0, k = objectCollection.length; i < k; i++) {
            var innerCollection = getInnerCollection(objectCollection[i], props);
            result = concatIfNotExists(result, innerCollection);
        }
        return result;
    }
};

From a once over :

This is a problem : var search = JSON.stringify(obj); because JSON.stringify() does not guarantee any order.. To be completely correct I think you are going to need to pass the properties to arrayObjectIndexOf that need to get compared ( that is, if uniqueness is determined by only comparing the provided properties, otherwise you are in a world of trouble )
"Inner property is not a collection!" <- Perhaps props[l-1] + " is not an array!", then you will provide which property is not an array, and be clear that you are expecting an array
"Property doesn't exist!" <- "Property " + props[j] + "does not exist" will give a better feedback as to which part is not working
JSHint.com could only find semicolon issues, which is good