word break using iteration Here is my word break problem in c#

static void breakSentenceIntoWords(string input,List<string> validWords)
    {
        if (String.IsNullOrEmpty(input))
            throw new Exception("String cannot be empty");

        int indicator = 0;
        StringBuilder builder = new StringBuilder();
        for (int i = 0; i < input.Length; i++)
        {
            builder.Append(input[i]);
            if (validWords.Contains(builder.ToString()))
            {
                indicator += builder.ToString().Length;
                Console.Write("{0} ", builder.ToString());
                builder.Clear();
            }
        }

        if (indicator != input.Length)
        {
            Console.WriteLine("Not all words could be found");
        }
    }

    List<string> validwords = new List<string>() { "I", "like", "had", "play","to" };
        breakSentenceIntoWords("Ihadplayliketo", validwords);

I think the complexity here is O(n2) because of the for-loop and the contains method. How can I improve its efficiency ?

Here you're using the name word but from your example you're actually working with substrings. Word is a vague concept but think about this:

var validWords = new List<string>() { "all", "to", "get", "her" };
BreakSentenceIntoWords("alltogether", validWords);

In this case you have each word from validWords but you also have another word together which is an English word but it's not in your list. Is it correct? If you're talking about words you need to consider some kind of delimiter (spaces, for example) but don't forget that it's not true in every language (Chinese, just to mention one). In your current code you're not even handling spaces.

Also note that in your current code order matters, for the above string you will get different results with { "all", "to", "together" } and this is (probably) unwanted. There are also some other corner cases you didn't handle (see answer below for other details.) I'd suggest that even before you start writing the first line of code you put in-place a huge test set (some of them also adding some randomness, I like my tests to be deterministic but working with text I always fear I forgot something then 10/20 iterations shuffling dictionaries and strings may help me to improve my test suite.)

Let's start with few general points, regardless what I said above about words.

Method name isn't best one you can choose because you're not breaking a sentence into words but you're checking if all words in that sentence are in a list of valid words. Two methods SplitStringIntoWords() and AreAllWordsKnown() might be more appropriate.

You're throwing Exception but you can do better than that, appropriate specific exception type is more informative and will help caller to handle such errors (you may have, for example, different policies for exception handling.) In this case I'd use ArgumentException. I'd also differentiate between a null input and an empty (or blank) input (using ArgumentNullException and ArgumentException respectively.) You should also check validWords to be not-null and not empty. Something like this:

if (input == null)
    throw new ArgumentNullException(nameof(input));

if (String.IsNullOrBlank(input))
    throw new ArgumentException("Input string cannot be blank", nameof(input));

if (validWords == null)
    throw new ArgumentNullException(nameof(validWords));

if (validWords.Count == 0)
    throw new ArgumentException("Known words list is empty.", nameof(validWords));

if (validWords.Any(x => x == null))
    throw new ArgumentException("Known words can't be null.", nameof(validWords));

Second step is to make this method more generic, you do not need caller to use List<string> when any IEnumerable<string> is enough to complete your task. Just change method signature.

You're also writing to console making this method not easy to reuse. Split logic and presentation, you may use a callback parameter if you need to output something or change return type to hold all the information you need. We will see more about this later.

First performance problem comes from iteration through the input string, for each character you search validWords for a match, this can be pretty inefficient when validWords or input grow to larger numbers. A small note about word searching. Linear search is not efficient for very large dictionaries (but it may be OK for 10~20 words), ordered lists and hashtables offer a huge performace gain in search (because you won't actually need to go through the entire list performing an expensive culture aware string comparison to find a match.) Later there is more about this...

Let's split your sentence into words! .NET Framework already has a function ready to use: String.Split(). If you do not specify any delimiter then it will use whitespace characters (which are many more than space and tab). For simplicity in these examples I will omit error checking (and I won't consider punctuation) but you need to do it in your code.

static string[] SplitIntoWords(string sentence)
{
    return sentence.Split(null);
}

You might want to compare with regex implementation to determine which one performs better:

static string[] SplitIntoWords(string sentence)
{
    return Regex.Split(sentence, "\\s+");;
}

Now that you have a list of words you can determine if all of them are valid. First naive implementation (don't forget what we said above about string searching in a dictionary) may be:

static bool AreAllWordsKnown(IEnumerable<string> words, IEnumerable<string> knownWords)
{
    return words.All(x => knownWords.Contains(x));
}

Note that to extract the list of not valid words you can use:

IEnumerable<string> unkownWords = words.Except(knownWords);

Above code can then be rewritten as:

return words.Except(knownWords).Any() == false;

To determine the list of valid words is straightforward (it's just the intersection between words and unknownWords.)

Things are different if you're not searching for words but for substrings. In this case, to avoid ordering problems, I'd first sort input list according to word length (to prefer longer matches):

var orderedValidWords = validWords.OrderBy(x => x.Length);

In this example I'm simply using String.Length but you know it's number of UTF-16 code units, not the number of characters of the string then you may have unexpected results. If you need a true culture aware check this is little bit more complex and slow:

var orderedValidWords = validWords.OrderBy(x => new StringInfo(x).LengthInTextElements);

Note that to perform a valid encoding-aware comparison you should enumerate text elements (which are...System.String) using StringInfo methods instead of System.Char.

What is slow in your actual code? You build a string for comparison, character by character. This wastes some time but think if all known words are at least 6 characters length (average word length in English is 5.1 characters), you will perform useless expensive searches. More than that you're copying characters (possibly increasing StringBuffer internal storage) just to create a new string each time with ToString() (twice...) You  may want to experiment with a custom ordinal comparison of char[] but you should first go with String.SubString (you may light to read this SO post about SubString). In short you just keep two indices (beginning of the word and current word length) and then you do:

validWords.Contains(input.SubString(wordBeginIndex, wordLength))

This alone will vastly improve performance but you can do better (for big dictionaries) adding some logic (experiment and measure) because now dictionary is ordered then:

You can check string length:

If shorter word has 4 characters and you current word is "get" then you don't need to perform any search.
If longer word is 5 characters and your actual word is 6 then (and you're not handling this in your actual code) you should move wordBeginIndex one character forward (note that this may be tricky if you want to keep track of unknown words, to simply report there are some is straightforward.)

You do not need to go through the whole list, if your word is "banana" and your list is "apple, cider, lemon" you can stop immediately when you see "cider". Be careful because ordering both for word length and alphabetically is tricky, you may want to keep only alphabetical order for simplicity (but measure, measure, measure...)
You can perform a bisection search.

What's a very fast way to search for a string in a list? HasShet<string>, even the generic implementation:

var dictionary = new HashSet<string>(validWords);

Note however that with hash-set ordering isn't preserved then you have the above mentioned ordering problem and you have to handle it elsewhere.


CodeIgniter Active Record Subqueries I use CodeIgniter at work, and one of our model files had a lot of subqueries in it.  I originally had to manually write each subquery, and wondered if I could use active records instead.
So, to make my life easier, I made a subquery library for CodeIgniter.
I put it on the CodeIgniter Wiki, but I never really had any one look over it.  So, can you tell me if there is anything I should improve in this, or anything I really shouldn't be doing?
P.S. Feel free to use this if you wish.
P.P.S. join_range is a helper method for use with the answer to this question.
P.P.P.S. The latest version can be found here.
class Subquery{
    var $CI;
    var $db;
    var $statement;
    var $join_type;
    var $join_on;

    function __construct(){
        $this->CI =& get_instance();
        $this->db = array();
        $this->statement = array();
        $this->join_type = array();
        $this->join_on = array();
    }

    /**
     * start_subquery - Creates a new database object to be used for the subquery
     *
     * @param $statement - SQL statement to put subquery into (select, from, join, etc.)
     * @param $join_type - JOIN type (only for join statements)
     * @param $join_on - JOIN ON clause (only for join statements)
     *
     * @return A new database object to use for subqueries
     */
    function start_subquery($statement, $join_type='', $join_on=1){
        $db = $this->CI->load->database('', true);
        $this->db[] = $db;
        $this->statement[] = $statement;
        if(strtolower($statement) == 'join'){
            $this->join_type[] = $join_type;
            $this->join_on[] = $join_on;
        }
        return $db;
    }

    /**
     * end_subquery - Closes the database object and writes the subquery
     *
     * @param $alias - Alias to use in query
     *
     * @return none
     */
    function end_subquery($alias=''){
        $db = array_pop($this->db);
        $sql = "({$db->_compile_select()})";
        $alias = $alias!='' ? "AS $alias" : $alias;
        $statement = array_pop($this->statement);
        $database = (count($this->db) == 0)
            ? $this->CI->db: $this->db[count($this->db)-1];
        if(strtolower($statement) == 'join'){
            $join_type = array_pop($this->join_type);
            $join_on = array_pop($this->join_on);
            $database->$statement("$sql $alias", $join_on, $join_type);
        }
        else{
            $database->$statement("$sql $alias");
        }
    }

    /**
     * join_range - Helper function to CROSS JOIN a list of numbers
     *
     * @param $start - Range start
     * @param $end - Range end
     * @param $alias - Alias for number list
     * @param $table_name - JOINed tables need an alias(Optional)
     */
    function join_range($start, $end, $alias, $table_name='q'){
        $range = array();
        foreach(range($start, $end) AS $r){
            $range[] = "SELECT $r AS $alias";
        }
        $range[0] = substr($range[0], 7);
        $range = implode(' UNION ALL ', $range);

        $sub = $this->start_subquery('join', 'inner');
        $sub->select($range, false);
        $this->end_subquery($table_name);
    }
}

Example Usage
This query:
SELECT `word`, (SELECT `number` FROM (`numbers`) WHERE `numberID` = 2) AS number
FROM (`words`) WHERE `wordID` = 3

would become:
$this->db->select('word')->from('words')->where('wordID', 3);
$sub = $this->subquery->start_subquery('select');
$sub->select('number')->from('numbers')->where('numberID', 2);
$this->subquery->end_subquery('number');

Personally I think your going the wrong way about things, you can easily pass in a query string into the select method and set the 2nd param to true to bypass backticks.
So the output would place the sub query string within the main query select.
I would do something along the lines of:
class MyModel extends Model
{
    public function getRows()
    {
        //Create a subquery and render it to a stirng
        $sub = $this->db->select('number')->from('numbers')->where('numberID', 2)->_compile_select();

        //Clear the data from the CI Arrays
        $this->db->_reset_select();

        //Build the main query passing in the sub-query and disabling backticks
        $this->db->select("word,(" . $sub . ")", false)->where('wordID', 3);

        //Get the results
        $result = $this->get("words");
    }
}

Sources:

@ _compile_select()
@ _reset_select()

Firstly let me just state that the code above may not be fully working as i have not test machine a.t.m, but I do know that this is possible and you do not need all the extra logic specified.
It seems pretty simple to me without creating new $db's.
I also would recommend you encapsulate the logic above into a class so you can pass the object's around and make life simpler as the above is a POC

Concept:
class InnerQuery extends CI_DB_active_record
{
    public function __construct()
    {
    }

    public function __call($method,$params = array())
    {
        //Remove methods that modify the database
        switch(strtolower($method))
        {
             case 'get':
             case 'count_all_results':
             case 'get_where':
                 trigger_error("Cannot use {$method} in InnerQuery");
             break;
        }
        return $this;
    }

    public function compile()
    {
        return "(" . $this->_compile_select() . ")";
    }

    public function __tostring()
    {
        return $this->compile();
    }
}

Ok so the above class extends the same object as $this->db in your controller, so you can use all the methods to build a query such as
$this->InnerQuery->select("item as item_key")->from("inner_table")->where("foo","zed");

You should disable the parent methods that change the database or run any queries as this is only used to build a select string.
so you should in thoery be able to do:
$this->db->select("word")->where('wordID', 3);
$this->db->select($this->InnerQuery,false);

which would use the DB class to build your query and can just be passed into the outer select and the __tostring will return the (SELECT ...) with braces and pass it into the main query.


Finding shortest distance between a point and a surface using Lagrange Multipliers I am new to Matlab. Now I need to find the shortest distance between a given point to a surface, which is describe with a function.

I am planing to implement the method described in the link below with Lagrange Multipliers, and have writen my code to do that. However, I found my code runs slowly.

https://math.stackexchange.com/questions/175624/finding-shortest-distance-between-a-point-and-a-surface

My code is as following:

% the function return the points with their distances to the surface (fun), and the corresponding points on the surface
function [ points_and_distance ] = getPoints( fun, var )
% m is the number of given points
m = 100;
% n is the dimension of variables, such as x, y, z, in my code I use x1, x2, x3, etc.
n = 3;
% given points are generated randomly, and this step is not the bottleneck
points = gen_points(m, n);
% l is used as the Lagrange Multiplier, an extra variable
syms l
fun_l = l * fun;
%expand var
var(n+1) = l;
% Each row of points_and_distance is the given point and the corresponding point on the surface with a shortest distance to the given point, and their distance, so the column number is 2*n+1
points_and_distance = zeros(m, 2*n+1);
tb=clock;
for i=1:m
    p = points(i, :);
    % Construct the Lagrangian function based on a given point
    for j=1:n
        fun_l = fun_l + (var(j) - p(j))^2;
    end
    tic;
    points_and_distance(i, :) = getDistance(fun_l, n+1, var, p);
    ct_each = toc;
    disp(['Get distance for ',num2str(i),'th point consumes: ',num2str(ct_each), 's']);
end
comsumedtime = etime(clock, tb);
disp(['Get distance for ',num2str(m),' points consumes totally: ', num2str(comsumedtime), 's']);
% sorting the result based on the distance column
points_and_distance = sortrows(points_and_distance, 4);
end

The definition of getDistance is as follows:

function [ rst ] = getDistance( func, n, x, x0 )
% Return Top-K records with shortest distance to given surface, which is described with func.
%
% input:
% Function func: the surface
% n: Number of variables in func
% x: Array of variables
% x0: given point
% Get Differentiation of each variable x(i)
% Each partial differentiation is stored in an array of functions
for i = 1:n
    eqs(i) = diff(func, x(i));
end
% Slove the equations
sol = solve(eqs, x);
% There may be multiple solutions. For each row of solutions, calculating the distance to x0
cur_sol = zeros(1, 4);
for r = 1:size(sol.x1, 1)
    min_dis = 0;
    t_dis = sqrt((sol.x1(r) - x0(1))^2 + (sol.x2(r) - x0(2))^2 + (sol.x3(r) - x0(3))^2)
    if min_dis == 0 || min_dis > t_dis
        % here I use n = 3, so there are three variables in func, which are represented with x1, x2, and x3.
        % In the fact, I did not find the way to return the solution of arbitrary variable, which may be represent as
        % sol.x(i) or something like that, where x is the array of symbolic variables. I have to hard code to use sol.x1, sol.x2.
        cur_sol(1) = double(sol.x1(r));
        cur_sol(2) = double(sol.x2(r));
        cur_sol(3) = double(sol.x3(r));
        min_dis = t_dis;
        cur_sol(4) = double(min_dis);
    end
end
% record the solution, i.e. the closest point on the surface, the distance to the given point x0, and x0
rst(1:4) = cur_sol(:);
rst(5:7) = x0(:);
end

I run the code hundreds of times, when m = 100 and n = 3, the average time consumed is about 300ms, when m = 100 and n = 10, the time can be 30s, which is too slow and not expected. I thought my code could be optimized to run faster. Since this is my first Matlab project and I have looked into the way to reduce loop, but still cannot improve it.

As far as I can tell what you do is a static optimization with constraints. Two comments on that:

1) Matlab provides fmincon to solve exactly this type of problem.

2) You use matlab, which is a numerical calculation program and make extensive use of symbolic variables. Your main loop manipulates a symbolic variable l. I suggest you research ways to replace the syms line completely. The direction I can see is using anonymous functions, ie something like

k1 = 3;
k2 = 10;
k3 = -7;
plane = @(x,y,z) k1*x+k2*y+k3*z;

this is a very powerful concept. You can set the parameters by other variables of course and in the end define your function (without symbolic variables!). You can even hand the function over to another function! e.g. solution = minimizerfun(plane,point);, and inside the minimizerfun you call the plane dist = plane(x,y,z) - start_point;! Surely, this can be vectorized plane = @(x) k1*x(1)+k2*x(2)+k3*x(3); which is close to what you should use here.

Mupad is some extension of matlab, mex is their term for externally compiled stuff (like C/Fortran programs compiled for matlab). My best bet is, that the symbolic toolbox uses some parts of this "external part". - But even if I'm wrong here, the toolbox is VERY slow compared to numerical calculation (which matlab is meant for)!


Merging two multidimensional associative arrays I just wrote a function that will loop through two multidimensional associative arrays and merge them together based on the key names and their values. While I'm satisfied that this function works, I have a strong feeling that it can be improved. Can I get some feedback?

<?php

function array_merge_defaults (array &$array1, array &$array2, $keyField)
{
    $merged = $array1;

    foreach ($array2 as $key => &$value)
    {
        $valueMerged = false;

        foreach ($merged as $mergedKey => &$item)
        {
            if (is_array($item) && array_key_exists($keyField, $item) && $item[$keyField] == $value[$keyField])
            {
                $item = array_merge($item, $value);
                $valueMerged = true;

                break;
            }
            else if ($mergedKey == $key)
            {
                if (is_numeric($mergedKey))
                {
                    $merged[] = $value;
                }
                else
                {
                    $item = $value;
                }

                $valueMerged = true;

                break;
            }
        }

        if (!$valueMerged)
        {
            $merged[$key] = $value;
        }
    }

    return $merged;
}

$defaults = array(
    array(
        "name" => "foo",
        "level" => 5,
        "meta" => array(
            "value" => 1
        )
    ),
    array(
        "name" => "bar",
        "level" => 6
    )
);

$custom = array(
    array(
        "name" => "qux",
        "date" => "2016-01-01"
    ),
    array(
        "name" => "tux",
        "date" => "2016-01-31"
    ),
    array(
        "name" => "foo",
        "data" => "2015-01-01",
        "meta" => array(
            "value" => 5,
            "something" => "hello world"
        )
    )
);

$merged = array_merge_defaults($defaults, $custom, 'name');

print_r($merged);

Here's how my final array looks

Array
(
    [0] => Array
        (
            [name] => foo
            [level] => 5
            [meta] => Array
                (
                    [value] => 5
                    [something] => hello world
                )

            [data] => 2015-01-01
        )

    [1] => Array
        (
            [name] => bar
            [level] => 6
        )

    [2] => Array
        (
            [name] => qux
            [date] => 2016-01-01
        )

    [3] => Array
        (
            [name] => tux
            [date] => 2016-01-31
        )

)

There is an alternative to your implementation. If you can change the position of the name element to the array key:

$defaults = array(
    'foo' => array(
        "level" => 5,
        "meta" => array(
            "value" => 1
        )
    ),
    'bar' => array(
        "level" => 6
    )
);

$custom = array(
    'qux' => array(
        "date" => "2016-01-01"
    ),
    'tux' => array(
        "date" => "2016-01-31"
    ),
    'foo' => array(
        "date" => "2015-01-01",
        "meta" => array(
            "value" => 5,
            "something" => "hello world"
        )
    )
);

You can utilize the native PHP function: array_replace_recursive

Using this function will provide you with the same resulting structure as you have shown with the exception of having the names as array keys. If this is an issue to you, you can duplicate the name once more into an array element.

The reason this specific/unique key is required is to only merge elements related to each other.

This little snippet should translate your array into the required structure:

/**
 * Translates an array element into an unique key for the element.
 *
 * (The name is just an example)
 */
function uniqify(array $array, string $key): array
{
    foreach($array as $index => $value)
    {
        $array[$value[$key]] = $value;
        unset($array[$index]);
    }
    return $array;
}

You could then so something like this:

$merged = array_replace_recursive(
    uniqify($defaults, 'name'),
    uniqify($custom,   'name')
);

Hope this can help, happy coding!


Safe way of creating unique IDs Is the code below safe to create unique ids for objects?

I will create new yoga classes and need to have a unique id for each class. I made this small piece of code to generate the id numbers.

Regarding specifically the uniqueIdCreatorHandler method, would this solution incur in bad performance, or is it just far from elegant?

what would you say?

CodeSandBox here

The code is also presented as per below:

import React from "react";

export default class App extends React.Component {
  state = {
    id: "",
    ids: [{ id: 7 }, { id: 14 }]
  };

  uniqueIdCreatorHandler = () => {
    let ids = [...this.state.ids];
    let highestId = 0;
    if (ids.length > 0) {
      highestId = ids
        .map(value => {
          return value.id;
        })
        .reduce((a, b) => {
          return Math.max(a, b);
        });
    }
    let newId = highestId + 1;
    ids.push({ id: newId });
    this.setState({ ids: ids });
  };

  idDeleterHanlder = currIndex => {
    let ids = this.state.ids;
    ids.splice(currIndex, 1);
    this.setState({ ids: ids });
  };

  render() {
    let ids = this.state.ids.map((id, index) => {
      return (

 this.idDeleterHanlder(index)} key={id.id}>
          id:{id.id}

      );
    });

    return (

        <button onClick={this.uniqueIdCreatorHandler}>Push new id</button>

all ids below:

        {ids}

    );
  }
}

I've noticed you have a common bug that often occurs in Javascript code called a race condition.

If uniqueIdCreatorHandler is called again before a previous call finishes there is a race condition created because state.ids is replaced at the end of the call.

If both calls happen at the same time they each generate a unique copy of the original state.ids then overwrite state this.setState({ ids: ids });. This causes the first change to state.ids to be forgotten.

This is a direct result caused by replacing state. this.setState({ ids: ids });

As others have recommend I would simply keep a increment in state for new ids.

Note: you can easily demonstrate this in your code by adding and calling this method

double = () => {
  this.uniqueIdCreatorHandler()
  this.uniqueIdCreatorHandler()
}

Other comments
There are small improvements you could make to your code like spreading the mapped ids into Math.max.

I wouldn't worry too much about Javascript performance, focus on readability. Make variable names more verbose and add doc blocks to methods.

One thing I like to see (which you have done) is the embrace of array methods. Although they may not be necessary here they are a strong indicator of comprehension.

Keep it up!


Loop through object and send analytics code based on objects properties value The code I have now is currently working, but I'm looking for ways to optimize it, and really, to just get any feedback.

I'm implementing this code through Ensighten (a tag management platform) as a custom javascript tag. Ensighten works by injecting tags either sync or async scripts on to a page. I've set this tag to fire after the DOM is ready.

What it does is creates a namespace, and then sets a function in that namespace to main (see main.js). The main function takes one argument, an object (see lookupTable.js), in which it takes that object, loops through the object, and if the object contains the correct key, it sets that key's value equal to campaignCode.

Next, the function looks to see if the campaignCode is not falsey, it then looks at the path (window.location.path) to see if it's either a courses or a topics page. If it's a topics page it fires the analytics code. If it's a courses page, it listens for a message from an iframe that a video was clicked. If the message from that iframe equals play button clicked then it fires the other analytics code.

The main function is called from the the lookupTable.js. Like so:

window.analytics.campaign.main(window.analytics.campaign.lookupTable);

Questions:

Should I be checking if the main function's argument is an object?
Is there a way I wouldn't have to check if the campaignCode is falsy? Could I check it in the if statement of the for loop?
Am I declaring the analyticsTrack function in the correct way?

main.js

/*jslint browser:true*/
/*globals Bootstrapper*/

window.analytics = window.analytics || {};
window.analytics.campaign = window.analytics.campaign || {};

window.analytics.campaign.main = function (lookupTable) {
    var path = window.location.pathname,
        campaignCode = '',
        prop;

    function analyticsTrack (id) {
        return Bootstrapper.insertScript('//www.example.com/click/services/Tracking/Click.ashx?CR_EAC=' + id);
    }

    for (prop in lookupTable) {
        if (lookupTable.hasOwnProperty(prop) && path === prop) {
            campaignCode = lookupTable[prop];
        }
    }

    // check campaignCode
    if (campaignCode) {

        // if it's a topic page, fire a RIO tag onload
        if (path.indexOf('topics') > -1) {
            analyticsTrack(campaignCode);
        }

        // if it's a courses page
        if (path.indexOf('courses') > -1) {

            // listen for a message from the iframe
            $(window).on('message', function (e) {

                // check the message from the iframe, fire a RIO tag
                if (e.originalEvent.data === 'play button clicked') {
                    analyticsTrack(campaignCode);
                }
            });
        }
    }
};

lookupTable.js

window.analytics = window.analytics || {};
window.analytics.campaign = window.analytics.campaign || {};

var campaign = window.analytics.campaign;

campaign.lookupTable = {
    '/topics/123456':    '300257956',
    '/courses/234567':   '300257957',
    '/courses/34567':    '300257958',
    '/courses/15677789': '300257959',
    '/topics/123457':    '300257970'
};

campaign.main(campaign.lookupTable);

Should I be checking if the main function's argument is an object?

No, document it as it accepts objects, the params are straight forward no need to check them for the consumer

Is there a way I wouldn't have to check if the campaignCode is falsy? Could I check it in the if statement of the for loop?

No need to loop just access the property directly

campaignCode = lookupTable[prop];

Am I declaring the analyticsTrack function in the correct way?

Yes and no, every time you invoke the main function you are allocating the method.  There are no closures around it so it doesn't need to be done.  Just move it outside of the main method but wrap everything in a self invoking method (if you are not already with some other tool) so it won't be global.

If I had to rewrite everything here is how I would do it.  Instead of having a bunch of if path contains x then do something with the campainCode statements I  just put it in an object.  It might be overkill but it is a good pattern to use in other places.  Say you want to handle books instead of adding another if indexof .. just add books to the pathsHandlers object.

(function() {

    function analyticsTrack(id) {
        return Bootstrapper.insertScript('//www.example.com/click/services/Tracking/Click.ashx?CR_EAC=' + id);
    }

    var pathHandlers = {
        topics: function(campaignCode) {
            analyticsTrack(campaignCode)
        },
        courses: function(campaignCode) {
            $(window).on('message', function(e) {

                // check the message from the iframe, fire a RIO tag
                if (e.originalEvent.data === 'play button clicked') {
                    analyticsTrack(campaignCode);
                }
            });
        }
    }

    window.analytics.campaign.main = function(lookupTable) {
        var path = window.location.pathname,
            campaignCode = lookupTable[path],
            handlerKey;

        if (campaignCode) {
            for (handlerKey in pathHandlers) {
                if (pathHandlers.hasOwnProperty(handlerKey) && path.indexOf(handlerKey) > -1) {
                    pathHandlers[handlerKey](campaignCode);
                }
            }
        }
    };

})();

--edit to answer comment

Your function was defined in the main function but it didn't need to be because it wasn't using any locally scoped variables in main (no variables being accessed in the function via closure).

If the method was for example :

window.analytics.campaign.main = function (lookupTable) {
    var path = window.location.pathname,
        campaignCode = '',
        prop;

    function analyticsTrack (id) {
        console.log('current path is ' + path)
        return Bootstrapper.insertScript('//www.example.com/click/services/Tracking/Click.ashx?CR_EAC=' + id);
    }

The method couldn't be moved outside of main because it is referencing the path variable within a closure.


Using events together with interfaces in VBA Introduction
Because of the limitation of VBA in using events in interfaces I was searching for a kind of workaround.
For sure I also read this which also provides an approach, but I was searching for an easier way.
I ended up with the following solution.

The idea behind
Instead of defining the events directly in an interface - because it is not possible to use them in an implementing class in VBA - I use an additional 'event' class, where all necessary events will be placed in, and which will be injected into the interface implementing classes.

Naming of the event class
I'm aware of that this class is not really used as an interface, but it only should be used with the regarding interface.
So I named it also with an I prefix.
Another benefit of this is that it will be listed beneath the regarding interface.

Circular references
The worker object is provided with the event on purpose.
As long as it is used in the event handler with care, that means, it should not be stored anywhere else, there shouldn't be any risk regarding circular references.

The interfaces

IWorker

Option Explicit

Public Property Set Events(ByRef value As IWorkerEvents)
End Property

Public Sub Work()
End Sub

IWorkerEvents

Option Explicit

Public Event Notify(ByRef worker As IWorker, message As String)

Public Sub Notify(ByRef worker As IWorker, message As String)
    RaiseEvent Notify(worker, message)
End Sub

The implementations

Worker1

Option Explicit

Implements IWorker

Private Type TWorker
    Events As IWorkerEvents
End Type

Private this As TWorker

Private Property Set IWorker_Events(RHS As IWorkerEvents)
    Set this.Events = RHS
End Property

Private Sub IWorker_Work()
    Debug.Print "Worker 1 works hard."
    Notify "is working..."
End Sub

Sub Notify(ByVal message As String)
    If Not this.Events Is Nothing Then
        this.Events.Notify Me, message
    End If
End Sub

Worker2

Option Explicit

Implements IWorker

Private Type TWorker
    Events As IWorkerEvents
End Type

Private this As TWorker

Private Property Set IWorker_Events(RHS As IWorkerEvents)
    Set this.Events = RHS
End Property

Private Sub IWorker_Work()
    Debug.Print "Worker 2 works hard."
    Notify "is working..."
End Sub

Sub Notify(ByVal message As String)
    If Not this.Events Is Nothing Then
        this.Events.Notify Me, message
    End If
End Sub

Building it together

TestClass

Option Explicit

Dim WithEvents workerEvents As IWorkerEvents

Sub Test()
    Dim worker As IWorker

    Set workerEvents = New IWorkerEvents

    Set worker = New worker1
    Set worker.Events = workerEvents
    worker.Work

    Set worker = New worker2
    Set worker.Events = workerEvents
    worker.Work
End Sub

Private Sub workerEvents_Notify(worker As IWorker, message As String)
    Debug.Print "TestClass says:", TypeName(worker), message
End Sub

TestModule

Option Explicit

Sub Test()
    With New testClass
        .Test
    End With
End Sub

Output

Worker 1 works hard.
TestClass says:             Worker1       is working...
Worker 2 works hard.
TestClass says:             Worker2       is working...

I use an additional 'event' class, where all necessary events will be placed in, and which will be injected into the interface implementing classes.

That. That is how it's done. It's COM-friendly, it works, and it's simple. Beautiful.

From Option Explicit to the names of literally everything, including that private type and that this field. You could open any of my own VBA projects and see exactly that, it's almost scary.

I only have a few minor points, that Rubberduck would have picked up:

Public access modifier is sometimes explicit, often implicit.
Not sure why IWorkerEvents parameter needs to be passed ByRef.
Not sure why Message parameter needs to be passed ByRef.

With the next Rubberduck release, you could also have @Description annotations, that the add-in automatically translates to VB_Description attributes, for example:

'@Description "Sets the event provider object for this instance."
Public Property Set Events(ByRef value As IWorkerEvents)
End Property

'@Description "Executes the worker."
Public Sub Work()
End Sub

These special comments (well, the actual description string) would then be visible in the object browser's bottom panel, and in Rubberduck's context-sensitive selection command bar, whenever an IWorker member is selected anywhere in the code.

The write-only Events property is also a little flag: it prompts for a better design - a factory method off a default instance comes to mind:

Option Explicit
'@PredeclaredId
Implements IWorker

Private Type TWorker
    Events As IWorkerEvents
End Type

Private this As TWorker

'@Description "Creates a new worker instance."
Public Function Create(ByVal workerEvents As IWorkerEvents) As IWorker
    If workerEvents Is Nothing Then Err.Raise 5, "IWorkerEvents instance cannot be Nothing."
    With New Worker1
        Set .Events = workerEvents
        Set Create = .Self
    End With
End Function

'@Description "Gets this instance through the IWorker interface. Used by the Create method."
Public Property Get Self() As IWorker
    Set Self = Me
End Property

'@Description "Gets or sets the worker events. Useless from default instance."
Friend Property Get Events() As IWorkerEvents
    Set Events = this.Events
End Property

Friend Property Set Events(ByVal value As IWorkerEvents)
    Set this.Events = value
End Property

Private Sub IWorker_Work()
    If this.Events Is Nothing Then Err.Raise 5, "Instance was not created with .Create method."
    Debug.Print "Worker 1 works hard."
    this.Events.Notify Me, "is working..."
End Sub

The IWorker interface then looks like this:

Option Explicit

Public Sub Work()
End Sub

So the client code is written against IWorker and only sees a Work method, and that's really all they need to care about.

Guard clauses ensure the object is always in a valid state and prevent misusing the class.

The Worker1 concrete class has a VB_PredeclaredId attribute, which exposes the Create method, so the code that New's up a Worker1 class can do this instead:

With Worker1.Create(workerEvents)
    .Work
End With

The fact that Events is visible from a Worker1 instance is not a problem, because the client code does not work from that interface, it only ever sees IWorker members; the Friend modifiers could just as well be Public, but there's no point exposing them beyond this VBAProject, so Friend is good enough.

Notice I removed the Notify procedure (which was implicitly Public), because it's really just an implementation detail that doesn't need to be exposed, and can very well simply be inlined in the Work method.

Until the next Rubberduck release, the module and member attributes need to be added manually, by exporting the module, editing them in, and then re-importing the module.

I like seeing test code. VBA code that works off interfaces and injected dependencies want to be tested! With Rubberduck you could have written unit tests that actually document the implementation/specs, for example in some Worker1Tests module:

Option Explicit
Option Private Module

'@TestModule
'@Folder "Tests"

Private Assert As New Rubberduck.AssertClass
Private Fakes As New Rubberduck.FakesProvider

'@TestMethod
Public Sub GivenNullWorkerEvents_Throws()
    Const ExpectedError As Long = 5
    On Error GoTo TestFail

    Dim sut As Worker1
    Set sut = New Worker1

    If Not sut.Events Is Nothing Then Assert.Inconclusive "Events should be Nothing"
    sut.Work

Assert:
    Assert.Fail "Expected error was not raised."

TestExit:
    Exit Sub
TestFail:
    If Err.Number = ExpectedError Then
        Resume TestExit
    Else
        Resume Assert
    End If
End Sub

This simple test passes when the Work method raises run-time error 5 because the object wasn't created with the Create function.

Another test would pass when the Work method would raise the Notify event on a fake implementation of the IWorkerEvents class, ensuring that the Worker1 class is calling the Notify member on the IWorkerEvents object it's given.

And so on: a robust test suite would cover pretty much every single execution path, and clearly document how the class should (and shouldn't) be used.

A single "here, see, it works" test doesn't really do anything other than exercising the "happy path" - which is ok for showing off a prototype, but production code needs all cases covered.


Clojure performance for solving Project Euler 72 (counting proper reduced fractions) Challenge

This is a solution of Project Euler 72 in Java.

  How may proper reduced fractions \$\dfrac{n}{d}\$ are there, where \$n
 < d \le 10^6\$?

Code

public long solve() {
  int limit = 1000000;
  int[] phi = IntStream.range(0, limit + 1).toArray();
  long result = 0;
  for (int i = 2; i <= limit; i++) {
    if (phi[i] == i) {
      for (int j = i; j <= limit; j += i) {
        phi[j] = phi[j] / i * (i - 1);
      }
    }
    result += phi[i];
  }
  return result;
}

The algorithm is explained in detail here. When I run the above code on my machine, it takes 150ms to get the answer.

I translated this code into Clojure like the below.

(defn solve []
  (let [limit 1000000
        phi (int-array (range (inc limit)))]
    (loop [i 2 acc 0]
      (if (= i (aget phi i))
        (loop [j i]
          (if (<= j limit)
            (do (aset phi j (/ (* (aget phi j) (dec i)) i))
                (recur (+ j i))))))
      (if (< i limit)
        (recur (inc i) (+ acc (aget phi i)))
        acc))))

This code uses Java array and mutate the value within the array. The algorithm is logically the same. However, when I run this code in repl, it takes over 25 seconds, which is a huge difference from the Java solution.

I expected that the Clojure code slightly slower than Java. But this is not a slight difference. Why is the Clojure code is slow like this. Did I miss something? Or is there other way to do this better?

(time (solve))

=> "Elapsed time: 27363.381633 msecs"

Replace aset with aset-int:

(defn solve []
  (let [limit 1000000
        phi (int-array (range (inc limit)))]
    (loop [i 2 acc 0]
      (if (= i (aget phi i))
        (loop [j i]
          (if (<= j limit)
            (do (aset-int phi j (/ (* (aget phi j) (dec i)) i))
              (recur (+ j i))))))
      (if (< i limit)
        (recur (inc i) (+ acc (aget phi i)))
        acc))))

(time (solve))

=> "Elapsed time: 443.570909 msecs"

This is still three times slower than the Java, but not out of sight.

I thought the original might be using type reflection, but ...

(set! *warn-on-reflection* true)

... produces no response to the original code.


What do you think of my Recursive FizzBuzz? I wanted to see if I fully understood recursion so I attempted the FizzBuzz challenge and applied recursion to it.

Did I do it correctly? Is this good code? Is there a more efficient way of doing it? How can I improve?

//---------------------------------------------------------------------------------------
//[RecurFizzBuzz]
//  FizzBuzz using recursion
//---------------------------------------------------------------------------------------
// Author : Jimmy
//---------------------------------------------------------------------------------------

public class RecurFizzBuzz {

 // Specify a range to recurse, starts at int a, ends at int b
 public static void recurse(int a, int b){

  if(a <= b){
   if(a % 3 == 0 && a % 5 == 0){
    System.out.println("FizzBuzz");
   } else if(a % 3 == 0){
    System.out.println("Fizz");
   } else if(a % 5 == 0){
    System.out.println("Buzz");
   } else {
    System.out.println(a);
   }
   recurse(++a, b);
  } else {
   System.exit(0);
  }
 }

 public static void main(String[]args) {
  recurse(1, 100);
 }
}

It's worth to mention a disadvantage of recursion: the possibility of stack overflow. Calling recurse(1, 6500) throws a StackOverflowError on my machine.
a % 3 == 0 && a % 5 == 0 could be a % 15 == 0. (Deleted, based on @QPaysTaxes's comment.)
recurse isn't a descriptive method name. printFizzBuzz would be better.
Use longer variable names for the parameters which explain the purpose, for example: lowerBound and upperBound.
They're easier to read and maintain.

  Without proper names, we are constantly decoding and reconstructing the
  information that should be apparent from reading the code alone.

(From codesparkle's former answer.)
System.exit isn't the nicest way to stop the recursion. You could use a simple return or omit the whole else block since the method returns anyway.


Number of occurence of two letters separted by a finit gap Lets we have a string Seq of size N and contain z alphabet { A,B,C….}.
I have to perform a function that calculates number of occurrence of letter i which is separated with l letters (gap) form the letter j:
Example:

I=A, j=T and gap=10
That have to find the number of occurrence of AT,A-T,A--T,A---T,A----T,,,A----------T(- is any alphabet)

I used this code for DNA sequence (alphabet z=4) and suppose that is N(L*N)
 complexity:

inline const short int V (const char x){
    switch(x){
    case 'A':case 'a':
      return 0;
      break;
    case 'C':case 'c':
      return 1;
      break;
    case 'G':case 'g':
      return 2;
      break;
    default:
      return 3;
    break;
    }

  }

  // [[Rcpp::export]]
  IntegerVector basem(std::string seq ,int gap ,int N){
    IntegerVector res(16);
     short int x=0;
    short int y=0;
    for(int i=0;i<N-gap;i++){
      x=V(seq[i]);
      y=V(seq[i+gap]);
      res[4*x+y]++;
    }
    return res;
  }

  // [[Rcpp::export]]
  List basegap (std::string sequence,int x){
    int n=sequence.size();
    List result ;
    for(int j=1;j<x;j++) {
      result.push_back(basem(sequence,j,n));
    }
    return result;
  }

  basem:calculate the number of occurence for a gap

  basegap: calculate the number of occurence for a gap from 1 to X

Running Example:

a
[1] "atgccatcactcagtaaagaagcggccctggttcatgaagcgttagttgcgcgaggactggaaacaccgctgcgcccgcccgtgcatgaaatggataacgaaacgccgaaaagccttattgctggtcatatgaccgaaatcatgcagctgctgaatctcgacctggctgatgacagtttgatggaaacgcggcatcgcatcgctaaaatgtatgtcgatgaaattttctccggtctggattacgccaatttcccgaaaatcaccctcattgaaaacaaaatgaaggtcgatgaaatggtcaccgtccgcgatatcactctgaccagcacctgtgaacaccattttgttaccatcgatggcaaagcgacggtggcctatatcccgaaagattcggtgatcggtctgtcaaaaattaaccgcattgtgcagttctttgcccagcgtccgcaggtgcaggaacgtctgacgcagcaaattcttgttgccgcgctacaaacgctgctgggcaccaataacgtggctgtctcgatcgacgcggtgcattactgcgtgaaggcgcgtggcatccgcgatgcaaccagtgccacgacaacgtcctctcttggtggattgttcaaatccagtcagaatacgcgccacgagtttctgcgcgctgtgcgtcatcacaactga"

 basem(a,5,nchar(a))
 [1] 40 50 42 37 47 46 36 49 45 39 44 39 38 42 45 28
x<-basegap(a,10)
[[1]]
 [1] 61 38 23 48 48 39 57 35 44 58 28 38 17 44 60 33

[[2]]
 [1] 42 47 43 38 42 49 53 35 44 44 35 44 42 39 37 36

[[3]]
 [1] 50 39 44 37 35 53 44 47 46 41 47 33 39 46 32 36

[[4]]
 [1] 42 54 36 38 54 36 52 36 43 49 41 34 31 39 38 45

[[5]]
 [1] 40 50 42 37 47 46 36 49 45 39 44 39 38 42 45 28

[[6]]
 [1] 42 44 40 42 45 44 44 45 38 44 47 38 44 45 36 28

[[7]]
 [1] 38 44 44 42 44 49 42 42 40 44 42 41 47 40 39 27

[[8]]
 [1] 34 48 47 38 46 49 41 41 47 39 39 42 42 40 40 31

[[9]]
 [1] 43 37 45 42 40 46 56 34 45 57 33 32 40 36 33 44

m<-matrix(unlist(x),nrow=16)
m
      [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9]
 [1,]   61   42   50   42   40   42   38   34   43
 [2,]   38   47   39   54   50   44   44   48   37
 [3,]   23   43   44   36   42   40   44   47   45
 [4,]   48   38   37   38   37   42   42   38   42
 [5,]   48   42   35   54   47   45   44   46   40
 [6,]   39   49   53   36   46   44   49   49   46
 [7,]   57   53   44   52   36   44   42   41   56
 [8,]   35   35   47   36   49   45   42   41   34
 [9,]   44   44   46   43   45   38   40   47   45
[10,]   58   44   41   49   39   44   44   39   57
[11,]   28   35   47   41   44   47   42   39   33
[12,]   38   44   33   34   39   38   41   42   32
[13,]   17   42   39   31   38   44   47   42   40
[14,]   44   39   46   39   42   45   40   40   36
[15,]   60   37   32   38   45   36   39   40   33
[16,]   33   36   36   45   28   28   27   31   44

My questions are:

How to reduce the complexity for this calculation?
Are there any ideas to improve this code?

I assume you are trying to find the number of pairs of a letter i and a letter j (i and j are arbitrary letters) that are at most L indices away from each other.

Use a sliding window algorithm. The sliding window will contain all indexes of an element i that are within the range of a certain index with the letter j.

Step 1. Store all indexes of the letter i in a sorted fashion (scan the input in order) O(N)

Step 2. Store all indexes of the letter j in a sorted fashion (similar to Step 1) O(N)

Example:

string s;  <- Assume input is already in this string
vector<int> letter1, letter2; <- Store indices
for (int i = 0; i < N; i++){
    if (s[i] == 'a'){         <- Our letter 'i'
        letter1.push_back(i);
    }
    if (s[i] == 'c'){         <- Our letter 'j'
        letter2.push_back(i);
    }
}

Step 3 (more complicated):

int track = 0, sum = 0;    <- Track the front-most index and the running sum of pairs
queue<int> window;  <- Our sliding window
for (int i = 0; i < letter2.size(); i++){   <- For every letter 'j'
    while (!window.empty() && window.front() < (letter2[i] - L)){
        window.pop_front();
    }
    while (letter1[track] <= (letter2[i] + L) && (letter1[track] >= (letter2[i] - L)){
        window.push(letter1[track]);
        track++;
    }
    sum = sum + window.size();
}

Let me explain some of the lines.

while (!window.empty() && window.front() < (letter2[i] - L)){
    window.pop_front();
}

The condition !window.empty() is to prevent a run-time error when calling window.front() later on. window.front() < (letter2[i] - L) implies that the smallest index in the queue is less than the minimum allowed value in the range, thus we should exclude it. We do not have to consider it ever again as the smallest possible value in the allowed range only increases as we iterate over letter2. We continue removing the elements till all of the elements are within the range or there are no elements left.

while (track < letter1.size() && letter1[track] <= (letter2[i] + L)  ){
    if ((letter1[track] >= (letter2[i] - L)) window.push(letter1[track]);
    track++;
}

The first condition track < letter1.size() is also to prevent a run-time error. The next condition is to check if the element is less than or equal the maximum allowed value in the range. If the element is also more than or equal to the smallest allowed value in the range, then we add it to the queue. We still increment track even if no value is pushed into the queue as at the end of each iteration of the loop, letter1[track] is always the smallest element larger than the allowed range during that iteration or is out of bounds.

All you have to do after this to output sum.

Although there are 2 nested loops, this algorithm runs in O(N) where N is the number of letters. Why? Because each index of the letter i is pushed and popped from the queue at most once each throughout the entire loop. This is at most 2*N operations, which is linear time.

All of the code is in C++ as that is my main language, though the concept can be applied to any language. You might want to look up other examples of the sliding window algorithm like this question from the Australian Informatics Olympiad.


Whose line is it anyway? I've just started delving into JavaFX, and the following is essentially my "Hello World". Although it's simple, I question the code formatting and wonder if I'm breaking any conventions, especially if explicitly concerning the library.

I also find myself concerned with what best promotes readability. Unfamiliarity brought many an alteration between styles. e.g. for statements, whether it is preferable to instantiate, modify and add an object all at once or simply pairing similar things together and modifying them wherever necessary and adding them at the end -- my final code here is an amalgamation of both styles.

This last bit may be delving a bit into SO/Programmers territory, but I'm curious.  I found myself using anonymous inner classes as it made methods easily transportable; I've hitherto not used it more than once in a program, is this bad form? If so, why?

import javafx.application.Application;
import javafx.beans.value.ObservableValue;
import javafx.scene.Group;
import javafx.scene.Scene;
import javafx.scene.control.Slider;
import javafx.scene.paint.Color;
import javafx.scene.shape.Line;
import javafx.scene.shape.StrokeLineCap;
import javafx.scene.text.Text;
import javafx.stage.Stage;

public class DrawingLines extends Application {
    public static void main(String[] args) {
        launch(args);
    }

    @Override
    public void start(Stage primaryStage) {
        primaryStage.setTitle("Legato's Lines");

        Group root = new Group();
        Scene scene = new Scene(root, 300, 150, Color.GRAY);

        Line redLine = new Line(10, 10, 200, 10) {
            {
                setStroke(Color.RED);
                setStrokeWidth(3);
                getStrokeDashArray().addAll(10d, 5d, 15d, 5d, 20d);
                setStrokeDashOffset(0);
            }
        };
        Line whiteLine = new Line(10, 30, 200, 30) {
            {
                setStroke(Color.WHITE);
                setStrokeLineCap(StrokeLineCap.ROUND);
                setStrokeWidth(10);
            }
        };
        Line blueLine = new Line(10, 50, 200, 50) {
            {
                setStroke(Color.BLUE);
                setStrokeLineCap(StrokeLineCap.BUTT);
                setStrokeWidth(5);
            }
        };

        Slider slider = new Slider(0, 100, 0) {
            {
                setLayoutX(10);
                setLayoutY(95);
            }
        };

        Text offsetText = new Text("Stroke Dash Offset: 0") {
            {
                setX(10);
                setY(80);
                setStroke(Color.WHITE);
            }
        };

        slider.valueProperty().addListener(
            (ov, curVal, newVal) -> offsetText.setText(
                "Stroke Dash Offset: " + Math.round(slider.getValue()))
        );
        redLine.strokeDashOffsetProperty().bind(slider.valueProperty());

        root.getChildren().add(redLine);
        root.getChildren().add(whiteLine);
        root.getChildren().add(blueLine);
        root.getChildren().add(slider);
        root.getChildren().add(offsetText);

        primaryStage.setScene(scene);
        primaryStage.show();
    }
}

It's hard to pick on such simple and straightforward code.
The one thing that stands out to me is the distance between the line where you declare the root variable, and the lines where you actually use it.
Let me quote an interesting paragraph from Code Complete:

  The code between references to a variable is a “window of vulnerability.” In the window, new code might be added, inadvertently altering the variable, or someone reading the code might forget the value the variable is supposed to contain. It’s always a good idea to localize references to variables by keeping them close together.

Indicators of this window of vulnerability are the measurements of variable "span" and "live time":

Variable "span": the number of lines between references to a variable. When a variable is referenced multiple times, the average span is computed by averaging all the individual spans. The smaller the better.
Variable "live time": the total number of statements over which a variable is "live". This is a count of lines between the first reference and the last. Again, the smaller the better.

By declaring root and scene at the top and using only at the button,
you have a large "window of vulnerability",
and high average spans and life times.
You can reduce these negative indicators by pushing these declarations down in your method to where the variables are actually used.


Fetching and storing new reddit posts I've created a small tool that fetches and stores the newest self-posts from a specific subreddit. The posts are stored in a redis store.

I'm going to write another couple of applications at a future date that will read from the redis store. I will want to be able to pick from the stored posts at random.

This is one of my first forays into writing something that's useful to me in rust, so I'd love some feedback.

#[macro_use]
extern crate serde_derive;

extern crate reqwest;
extern crate serde;
extern crate serde_json;
extern crate redis;

use redis::PipelineCommands;

#[derive(Deserialize)]
struct RedditResponse {
    data: Data,
}

#[derive(Deserialize)]
struct Data {
    children: Vec<Post>,
}

#[derive(Deserialize)]
struct Post {
    data: PostData,
}

#[derive(Deserialize)]
struct PostData {
    id: String,
    is_self: bool,
    title: String,
}

fn main() {
    let self_posts = match get_posts(20) {
        Ok(v) => v,
        Err(e) => panic!("Failed to fetch new posts: {:?}", e),
    };

    let redis_client = match redis::Client::open("redis://127.0.0.1") {
        Ok(c) => c,
        Err(e) => panic!("Unable to create the store client: {:?}", e),
    };

    let redis_conn = match redis_client.get_connection() {
        Ok(c) => c,
        Err(e) => panic!("Unable to retrieve the connection to the store: {:?}", e),
    };

    match add_posts_to_store(redis_conn, self_posts) {
        Ok(n) => println!("Stored {} posts.", n),
        Err(e) => panic!("Could not store posts: {:?}", e),
    };
}

fn get_posts(num: usize) -> reqwest::Result<Vec<Post>> {
    let url = format!("https://www.reddit.com/r/dota2/new.json?sort=new&limit={}", num);
    let posts: RedditResponse = reqwest::get(&url)?.json()?;

    Ok(
        posts.data.children
            .into_iter()
            .filter(|p| p.data.is_self)
            .collect::<Vec<Post>>()
    )
}

fn add_posts_to_store(store: redis::Connection, posts: Vec<Post>) -> redis::RedisResult<usize> {
    let mut pipe = redis::pipe();

    for p in &posts {
        pipe.set(&p.data.id, &p.data.title).ignore();
    }

    pipe.query(&store)?;

    Ok(posts.len())
}

The pattern

match some_result {
    Ok(v)  => v,
    Err(e) => panic!("message: {:?}", e),
}

is exactly what expect does for you. We can get a somewhat leaner main if we use that instead:

fn main() {
    let self_posts = get_posts(20).expect("Failed to fetch new posts");

    let redis_client =
        redis::Client::open("redis://127.0.0.1").expect("Unable to create the store client");

    let redis_conn = redis_client
        .get_connection()
        .expect("Unable to retrieve the connection to the store");

    match add_posts_to_store(redis_conn, self_posts) {
        Ok(n) => println!("Stored {} posts.", n),
        Err(e) => panic!("Could not store posts: {:?}", e),
    };
}

Somewhat, because the error messages lead to long lines and rustfmt doesn't like that.

However, keep in mind that while panic is great for developers, it's a system that should be used for unrecoverable errors, bugs that you cannot handle or anticipate. When you panic!, you generate an error message that's helpful for programmers, but not for users:

thread 'main' panicked at 'Unable to retrieve the connection to the store: .......', src/main.rs:.....
note: Run with `RUST_BACKTRACE=1` for a backtrace.

Instead, stderr should get used. The Rust Book contains a section for error handling in its example I/O project which you can use for inspiration. But if you never intend to distribute your reddit/redis updater, feel free to use panic! or .expect("...").

Other than that, the turbofish syntax at collect::<Vec<Post>>() isn't necessary, since its inferred by the return type. Also, you can traverse p in posts instead of &p in &posts since you don't want to use the Vec afterwards, you just have to save the posts.len() beforehand.

However, you probably want to use store : &redis::Connection in add_posts_to_store if you want to reuse the connection. Also, you usually want to derive some of the standard traits, for example Debug. Again, that depends on whether you will use your structs in another context, but if you ever publish your package it's very recommended to derive all possible standard traits (Debug, PartialEq, Eq, PartialOrd, Ord, Clone and maybe Hash in your case).


Parsing XML data to be put onto a site with jQuery I will preface by saying that this data set is much larger than what I have here. I'm having to loop through an XML file to get data and display it on a site with jQuery. I have to use jQuery because I'm working within a framework that I can't edit so I'm having to manipulate the data after the fact. This is causing the site I'm working on to slow significantly. I know that that's just the nature of what I'm doing but I'm running a loop inside a loop which I know can be pretty slow, especially when deal with 1,000 entries in an XML file. I'm curious if there's a way for me to do this in a more efficient way.

I'm basically looping through the data and on each iteration I'm grabbing that entries attributes and then having to loop through all the data within that iteration to compare the data with the rest of the data in the file. It seems like there's a better way to do this but I can't seem to figure it out.

If you need more clarification, let me know and I will provide it. I've pared the data down quite a bit so it's easier to handle and look at.

My XML is set up like this:

<Data>
    <Category>
        <category_number>1</category_number>
        <parent_number>4</parent_number>
        <category_name>This is a test Category</category_name>
    </Category>
    <Category>
        <category_number>2</category_number>
        <parent_number>3</parent_number>
        <category_name>This is another test Category</category_name>
    </Category>
    <Category>
        <category_number>3</category_number>
        <parent_number>4</parent_number>
        <category_name>This is yet another test Category</category_name>
    </Category>
</Data>

My HTML is set up like this:

This is a parent category. Category 4. A new list will be put in here with this categories child categories.

This is a parent category. Category 2. A new list will be put in here with this categories child categories.

My jQuery is as follows:

$.ajax({
    url: "/sample_url.xml",
    type: "GET",
    dataType: "html",
    success: function(data) {
        var xml = $.parseXML(data);
        $(xml).find('category_number').each(function(){
            category_number = $(this).text();
            var parent_number = $(this).parent().find('parent_number').text();
            var category_name = $(this).parent().find('category_name').text();

            var sub_array = [];

            //GET CATEGORIES SUB-CATEGORIES
            $(this).closest('Data').find('parent_number').each(function(){
                if($(this).text()==category_number){
                    var category_id = $(this).parent().find('category_number').text();
                    sub_array.push('
' + $(this).parent().find('category_name').text() + '
');
                }
            });

            $('ul#category_list').each(function(){
                if($(this).attr('class')==parent_number){
                    $(this).append('

');
                    $('ul#' + category_number).html(sub_array.join(""));
                }
            });
        });
    }
});

Don't use selector all the time. DOM locating waste most resource and time. as less selector as possible. use variable cache the jQuery object.

Data is look like a root:

var $data = $("Data");

Rather than parse start with "category_number", why not start with "Category"

$(xml).find('Category').each(function(){
    //do something.
});

Also, you could use filter() or map() to reduce the code:

// filter() example
function getSubCategory(pid){
    $catalist = $data.find("Category").filter(function(){
       return $(this).find("parent_number").text()==pid;
    })

    return $catalist;

}

// map() example
var subCatString=$subCatList.map(function(sum,item){
    var $subCat=$(item) ;
    var vcum=$subCat.find("category_number").text();
    return '
'+vcum+'
'
}).get().join();

Combine them together (HTML, see this JSFiddle, don't use AJAX):

var $xml= $("xml");
var $data=$xml.find("Data");
var $displayList=$("ul#category_list");

$xml.find("Category").each(function(){
    var $catageory=$(this);
    var $cnum=$catageory.find("category_number");
    var $pnum=$catageory.find("parent_number");
    var $cname=$catageory.find("category_name");

    var $subCatList=getSubCategory($cnum.text());
    var subCatString=$subCatList.map(function(sum,item){
        var $subCat=$(item) ;
        var vcum=$subCat.find("category_number").text();
        return '
'+vcum+'
'
    }).get().join();

    var $ul=$("
",{class:$cnum.text()});

    $ul.append(subCatString);

    $displayList.append($("
").html($ul));

});

function getSubCategory(pid){
    $catalist = $data.find("Category").filter(function(){
       return $(this).find("parent_number").text()==pid;
    })

    return $catalist;

}


Most elegant way to write this loop I have some code which looks like this:

do {
    hr = pEnumMgmt->Next(1, &mgmtObjProp, &ulFetched);
    if (hr == S_FALSE) { break; }
    llCbStorage += mgmtObjProp[0].Obj.DiffArea.m_llAllocatedDiffSpace;
} while (hr != S_FALSE);

It is bad because hr == S_FALSE is checked twice on every iteration.

I could write it like this:

while (true) {
    hr = pEnumMgmt->Next(1, &mgmtObjProp, &ulFetched);
    if (hr == S_FALSE) { break; }
    llCbStorage += mgmtObjProp[0].Obj.DiffArea.m_llAllocatedDiffSpace;
};

It is bad because it gives the impression that there exists no loop invariant (i.e. it could be an infinite loop, which would be a programmer error).

Finally, I could write it with gotos, but that is bad because gotos are considered harmful. Also, it gets really hard to check the correctness of the loop.

How would you write this loop and why?

My initial thought is to simply lose the break.  While they have their place in switch-case statements, it is generally best to avoid them in loops unless they're used at the top of a loop and truly help improve readability.  Even then, I tend to look for alternatives.

do {
    hr = pEnumMgmt->Next(1, &mgmtObjProp, &ulFetched);
    if (hr != S_FALSE) {
        llCbStorage += mgmtObjProp[0].Obj.DiffArea.m_llAllocatedDiffSpace;
    }
} while (hr != S_FALSE);

Alternatively, you could just initialize hr prior to the while loop and avoid the extra check against S_FALSE every loop.

hr = pEnumMgmt->Next(1, &mgmtObjProp, &ulFetched);
while (hr != S_FALSE) {
    llCbStorage += mgmtObjProp[0].Obj.DiffArea.m_llAllocatedDiffSpace;
    hr = pEnumMgmt->Next(1, &mgmtObjProp, &ulFetched);
}

I don't like the following style of while loop as it utilizes a side-effect.

while ((hr = pEnumMgmt->Next(1, &mgmtObjProp, &ulFetched)) != S_FALSE) {
    llCbStorage += mgmtObjProp[0].Obj.DiffArea.m_llAllocatedDiffSpace;
}

Lastly, it may be worthwhile to consider a for loop here.  I like the look of this because it cleanly separates the looping from the updates to llCbStorage.

for (hr = pEnumMgmt->Next(1, &mgmtObjProp, &ulFetched);
     hr != S_FALSE;
     hr = pEnumMgmt->Next(1, &mgmtObjProp, &ulFetched))
{
    llCbStorage += mgmtObjProp[0].Obj.DiffArea.m_llAllocatedDiffSpace;
}


Chinese mapping number routing I want to design Chinese mapping number routing. How can I improve this code in the method mapRouting?

I want to use reflect auto get method name by providing a value without if else, but I have no idea how to do it.

public class FwMappingChi {

    public static Map<String, String> mapRouting(String mapName){
        if(mapName == "ACTION") {
            return ACTION;
        }
        else if(mapName == "ENVIRONMENT") {
            return ENVIRONMENT;
        }
        else if(mapName == "RULE_TYPE") {
            return RULE_TYPE;
        }
        return null;
    }

    @SuppressWarnings("serial")
    private static final Map<String, String> ACTION = new HashMap<String, String>(){
        {
            put("0", "新增");
            put("1", "刪除");
        }
    };

    @SuppressWarnings("serial")
    private static final Map<String, String> ENVIRONMENT = new HashMap<String, String>(){
        {
            put("0", "正式環境");
            put("1", "長期UAT");
            put("2", "測試需求");
        }
    };

    @SuppressWarnings("serial")
    private static final Map<String, String> RULE_TYPE = new HashMap<String, String>(){
        {
            put("0", "系統間連接");
            put("1", "系統間連接");
            put("2", "人員維護用");
            put("3", "提供服務用");
            put("4", "User對外連接");
        }
    };
}

example:

List<Map<String, Object>> exportDatas = fwApplyListDetailMapper.getExportQuery(idsArr);
List<List<String>> pdfValues = new ArrayList<>();
for (int i = 0; i < exportDatas.size(); i++) {
    List<String> record = new ArrayList<>();
    Map<String, Object> map = exportDatas.get(i);
    for (String index : headerIndex) {
        Object value = map.get(index);

        if(FwMappingChi.mapRouting(index) != null) {
            record.add(FwMappingChi.mapRouting(index).get(String.valueOf(value)));
        }

    }
    pdfValues.add(record);
    record = null;
}

You could simply put your maps in a map, as the mapRouting function is nothing but a simple lookup.

Prepare somewhere

private static final Map<String, Map<String, String>> MASTER = new HashMap<>() {
   {
       put("ACTION", ACTION);
       put("ENVIRONMENT", ENVIRONMENT);
       put("RULE_TYPE", RULE_TYPE);
   }
};

And simply access that map:

public static Map<String, String> mapRouting(String mapName) {
    return MASTER.get(mapName);
}

BTW: in times of Java 10 you should really use Map.of instead of anonymous map subclasses.


Strategy for avoiding threadpool starvation while performing cpu bound jobs in a queued fashion My aim is to avoid using threadpool threads for CPU bound work, thus avoiding a situation where IIS stops responding to new requests.

Can you see any problems with the code below? Is this a safe/clean approach? Can you offer any improvements?

    private static ConcurrentQueue<Job> Jobs = new ConcurrentQueue<Job>();
    static int threadCount = 0;

    private void QueueJob(Job job)
    {

        bool startQueue=false;

        lock(Jobs)
        {
            if (threadCount == 0)
            {
                Interlocked.Increment(ref threadCount);
                startQueue = true;
            }
        }

        Jobs.Enqueue(job);

        if (startQueue)
        {
           var t= new Thread(new ThreadStart(ConsumeQueue));
           t.Start();
        }

    }
    private void ConsumeQueue()
    {
        while (true)
        {
            lock (Jobs)
            {
                if (!Jobs.Any())
                {
                    Interlocked.Decrement(ref threadCount);
                    return;
                }
            }

            Job j;

            var jobToDo = Jobs.TryDequeue(out j);

            if (jobToDo)
            {
                DoCPUBoundWork(j);
            }
        }

    }

I think a much easier solution is to just increse the number of threads available in the thread pool by calling ThreadPool.SetMaxThreads().

Also, if you have this many threads, then it means that:

either you're not going to get any performance improvements from parallelization, because all your cores are busy
or most of your threads are blocked, which means a better approach would be to actually solve the problem by using asynchronous waiting (possibly with the help of C# 5 async-await), instead of working around the problem.


Travelling Salesman in Qt I am writing a recursive function based on the Travelling Salesman Problem.

Is this a correct way of doing things, or am I creating memory leaks?

Prerequisites are:

a two dimensional matrix distance_matrix[][] to store the distances between the cities
a global int called current_minimum_cost which will be MAX_INT for starters (or any high 'impossible' number), it keeps track of the cost of the current 'optimal' path
remaining_cities: a QList of ints which at starting point contains all the cities to visit (except the HOME_CITY) (every city is represented by an integer, the same integer is used to look up the distance between two cities by using distance_matrix[cityA][cityB].
path: a QList of ints which stores the current path followed (so 0 , 7, 2, 5) means the salesman goes from his HOME_CITY to city #7 to city #2 to city #5 . At starting point it only contains the HOME_CITY (which is 0)
after doing his tour, the salesman has to return to his HOME_CITY (0)

void tsp(QList<int>* path, int total_cost, QList<int>* remaining_cities)
{
    int last_visited_city = path->last();

    if(total_cost > current_minimum_cost)
    {
        //cost higher than current_minimum_cost => ABORT
        return;
    }

    if(remaining_cities->empty())
    {
       //all cities visited

       //only add the distance back to the home city (salesman returns home)
       //path += HOME_CITY en total_cost += terug naar HOME_CITY";
       path->push_back(HOME_CITY);

       total_cost += distance_matrix[last_visited_city][HOME_CITY];

       if (total_cost <= current_minimum_cost)
       {
           // we have a new winner!
           current_minimum_cost = total_cost;

           // forget about the previous 'winner' and save the visited path
           tsp_solution->clear();
           foreach(int element , *path)
               tsp_solution->push_back(element);
       }
       // if not, than it means that "returning home" makes the trip more costly than the previous 'winner' (we always 'return home', so this is also taken into account with the previous 'winner')
       return;
    }

    // calculate the new possibilities to travel to from the current city:
    foreach(int next_city_to_visit , *remaining_cities)
    {
      QList<int>*newPATH = new QList<int>();
      newPATH->append(*path);
      newPATH->push_back(next_city_to_visit);

      int total_cost_for_this_new_path = total_cost + distance_matrix[last_visited_city][next_city_to_visit];

      // make a list of 'remaining cities' to visti:
      QList<int>* newREMAINING_CITIES = new QList<int>();
      newREMAINING_CITIES->append(*remaining_cities);

      // of course we do not have to visit 'next_city_to_visit' twice
      newREMAINING_CITIES->removeOne(next_city_to_visit);

      // recursion:
      tsp ( newPATH , total_cost_for_this_new_path , newREMAINING_CITIES );
      delete newPATH;
      delete newREMAINING_CITIES;
    }
}

My questions:

Does this function make sense? It seems to work for all my test cases.
Once I go up to 20 cities, it takes a long while to calculate, over an hour.
On memory management: is issuing delete newPATH enough to clear the memory? Or should I issue a delete for all members of the list newPATH? (I guess not as the members are all ints)

I'd have serious doubts about its being the fastest possible code, but just for fun let's consider the simplest possible code to generate correct answers.

The standard library has next_permutation, which can generate all the possible permutations of a collection. So, let's consider how we do this. We start with a 2D array of distances between cities. We then have a collection to represent a route through the cities. Since each city must appear once in the collection, we can start with it just containing numbers from 0 to N in order. We then permute those, find the distance for each, keep the shortest, and we're (eventually) done.

#include <algorithm>
#include <vector>
#include <iostream>
#include <numeric>
#include <limits>

static const int max = 13;

void setup(int distances[max][max]) {
    for (int i = 0; i < max; i++) {
        distances[i][i] = 0;
        for (int j = i + 1; j < max; j++) {
            int d = rand() % 50;

            distances[i][j] = d;
            distances[j][i] = d;
        }
    }
}

int main() {
    int distances[max][max];

    setup(distances);

    std::vector<int> route(max);

    std::iota(route.begin(), route.end(), 0);

    int best_distance = std::numeric_limits<int>::max();
    do {
        int distance = 0;
        for (int i = 1; i < route.size(); i++) {
            distance += distances[route[i - 1]][route[i]];
            if (distance > best_distance)
                break;
        }
        distance += distances[route[max - 1]][0];

        if (distance < best_distance) best_distance = distance;
    } while (std::next_permutation(route.begin() + 1, route.end()));

    std::cout << "best distance: " << best_distance << "\n";
}

Now some might wonder why I'd question this being the fastest way to do the job. Generic algorithms are, after all, intended to provide nearly the speed of custom-written code--their runtime overhead is usually quite minimal.

While that's true, in this case an important optimization is missed. If we view the routes as a tree or a graph, we can stop traversing a branch as soon as we reach a node where the current distance is greater than the best distance so far. This can avoid visiting a large number of paths based on a single test.

Using next_permutation, however we don't get that optimization. Since we don't know the relationships between paths, information we obtain from traversing one path can't be used to eliminate other possible paths.

While probably pretty fast for small sets of data, this is likely to be substantially slower than many others when N gets large at all.

Note: this does not depend upon the triangle inequality (and the test data it generates almost certainly doesn't follow the triangle inequality either).


Fast affine transformations of many 3D points by one 3×4 matrix I wrote a function to batch-transform 3D vectors by a single 3x4 matrix using SSE2:

struct alignas(16) Matrix3x4f
{
    union
    {
        Vector4 r[3];   // (r[0].w, r[1].w, r[2].w) contains the translation
        float   m[3][4];
    };
    // (0, 0, 0, 1) is the forth row of the full 4x4 matrix
};

struct Positions_SoA
{
    Vector4 *   xs;
    Vector4 *   ys;
    Vector4 *   zs;
};

void transform_points_SSE2(
    const Matrix3x4f& transform_,
    const Positions_SoA& inputs_,
    const int number_of_packets_,
    Positions_SoA &outputs_
    )
{
    //FIXME: woudn't this cause too much register pressure and memory loads?
    // For AVX/AVX2 we'll have to splat 8/16 values.
    const Vector4 m00 = SPLAT_X( transform_.r[0] );
    const Vector4 m01 = SPLAT_Y( transform_.r[0] );
    const Vector4 m02 = SPLAT_Z( transform_.r[0] );
    const Vector4 m03 = SPLAT_W( transform_.r[0] ); // translation X

    const Vector4 m10 = SPLAT_X( transform_.r[1] );
    const Vector4 m11 = SPLAT_Y( transform_.r[1] );
    const Vector4 m12 = SPLAT_Z( transform_.r[1] );
    const Vector4 m13 = SPLAT_W( transform_.r[1] ); // translation Y

    const Vector4 m20 = SPLAT_X( transform_.r[2] );
    const Vector4 m21 = SPLAT_Y( transform_.r[2] );
    const Vector4 m22 = SPLAT_Z( transform_.r[2] );
    const Vector4 m23 = SPLAT_W( transform_.r[2] ); // translation Z

    for( int i = 0; i < number_of_packets_; i++ )
    {
        outputs_.xs[i] = V4_ADD(
            V4_ADD(
                V4_MUL( m00, inputs_.xs[i] ),
                V4_MUL( m01, inputs_.ys[i] )
            ),
            V4_ADD(
                V4_MUL( m02, inputs_.zs[i] ),
                 m03
            )
        );

        outputs_.ys[i] = V4_ADD(
            V4_ADD(
                V4_MUL( m10, inputs_.xs[i] ),
                V4_MUL( m11, inputs_.ys[i] )
            ),
            V4_ADD(
                V4_MUL( m12, inputs_.zs[i] ),
                m13
            )
        );

        outputs_.zs[i] = V4_ADD(
            V4_ADD(
                V4_MUL( m20, inputs_.xs[i] ),
                V4_MUL( m21, inputs_.ys[i] )
            ),
            V4_ADD(
                V4_MUL( m22, inputs_.zs[i] ),
                m23
            )
        );
    }
}

The complete, runnable code is on rextester.

When I run the tests and compare the numbers with the reference (scalar) version, the results of the SSE version appear slightly wrong in half of all the cases, although the printed numbers look the same. Is that due to rounding errors? (FPU internally uses 80-bit precision, and SSE units compute with 32-bit precision.) Should I ignore those small discrepancies? Is my 'optimized' function even correct?
Why is the speedup due to SSE so small? I expected 3-3.5 speed increase, but got less than 2. Should I use AVX?
I plan to transform up to 64 points, is it still worth to use SSE/AVX?
Will performance be dominated by the cost of loading into registers?
I'm worried about the 12 shuffle/broadcast/replicate instructions.
Finally, I'll appreciate comments on my programming style, my mistakes in using SSE, a note on best practices and performance pitfalls.

Small differences due to precision are expected and can usually be ignored.

12 shuffles like that are a bit much, though not necessarily avoidable, depending on whether there is AVX support. With AVX, it is better to literally broadcast from memory, rather than emulate broadcasting with a load and shuffles. Even though this means there will be more loads, the shuffles are a bigger problem: loads (even broadcasting loads) typically have a throughput of 2/cycle while the shuffles typically have a throughput of 1/cycle. This means that _mm_set1_ps(float) is actually the better option, as long as you compile with AVX support. That wouldn't use the wider width yet though.

Also annoying is that MSVC apparently likes to reload those pointers inputs_.xs etc continuously. There is really no spare load-throughput for that, despite the presence of 9 FP additions (which are also bad though, since they're 1/cycle pre-Skylake), there are 21 loads there, limiting the throughput of the loop to once every 10.5 cycles (which is more than 9, if barely). Fortunately it can very easily be convinced to stop doing that, simply by copying them into local variables before the loop.

Next, I don't know what MSVC is up to there with its refusal to contract addition and multiplication into FMA, for processors that support it it should definitely be used. Using the corresponding intrinsic does the trick of course, but that makes it harder to compile for pre-FMA targets (Ivy Bridge and older). Anyway if I write the high level code like this:

Vector4 *   xs = inputs_.xs;
Vector4 *   ys = inputs_.ys;
Vector4 *   zs = inputs_.zs;
Vector4 *   outxs = outputs_.xs;
Vector4 *   outys = outputs_.ys;
Vector4 *   outzs = outputs_.zs;

for( int i = 0; i < number_of_packets_; i++ )
{
    outxs[i] = _mm_fmadd_ps(m02, zs[i],
               _mm_fmadd_ps(m01, ys[i],
               _mm_fmadd_ps(m00, xs[i], m03)));

    outys[i] = _mm_fmadd_ps(m12, zs[i],
               _mm_fmadd_ps(m11, ys[i],
               _mm_fmadd_ps(m10, xs[i], m13)));

    outzs[i] = _mm_fmadd_ps(m22, zs[i],
               _mm_fmadd_ps(m21, ys[i],
               _mm_fmadd_ps(m20, xs[i], m23)));
}

The actual code looks pretty good, apart from the fact that it uses short vectors:

$LL4@transform_:
    lea      rcx, QWORD PTR [rdx+rax]
    lea      rax, QWORD PTR [rax+16]
    vmovups xmm0, xmm7
    vfmadd231ps xmm0, xmm4, XMMWORD PTR [rax-16]
    vfmadd231ps xmm0, xmm5, XMMWORD PTR [r10+rax-16]
    vfmadd231ps xmm0, xmm6, XMMWORD PTR [r11+rax-16]
    vmovups XMMWORD PTR [rcx+rbx], xmm0
    vmovups xmm0, xmm3
    vfmadd231ps xmm0, xmm8, XMMWORD PTR [rax-16]
    vfmadd231ps xmm0, xmm9, XMMWORD PTR [r10+rax-16]
    vfmadd231ps xmm0, xmm10, XMMWORD PTR [r11+rax-16]
    vmovups XMMWORD PTR [rcx], xmm0
    vmovups xmm0, xmm1
    vfmadd231ps xmm0, xmm11, XMMWORD PTR [rax-16]
    vfmadd231ps xmm0, xmm12, XMMWORD PTR [r10+rax-16]
    vfmadd231ps xmm0, xmm13, XMMWORD PTR [r11+rax-16]
    vmovups XMMWORD PTR [r9+rax-16], xmm0
    sub      r8, 1
    jne      SHORT $LL4@transform_

That doesn't really address the SSE2 side of the store directly, but you can do the same thing but with separate add/mul. Proper use of AVX would use the wider vectors. For this code that's a fairly trivial change, just make almost everything wider (except the matrix). If AVX is available, you should use it in this case - it doesn't help for everything, but for this type of code (almost purely vertical SIMD, apart from some broadcasts) it's great. FMA is also a free performance win here (especially on Haswell and Broadwell), well worth using even if it means writing two versions and doing runtime dispatch to also support the Bridges (which are perhaps not old enough yet to completely disregard).

Supporting 32bit machines is annoying. They don't even have enough registers to load all of that matrix, so everything gets bogged down by the extra loads that are suddenly required. I don't see good fixes. Rearranging the multiplication so that the matrix can be held in 4 registers does work, but then a horizontal addition appears which is bad, and the multiplication wastes a lane on padding. I expect it would be worse than the extra loads, on the other hand if we're talking about old hardware like Core2 (which had x64 support, but at the time installing a 64bit OS was a rarity) then extra loads are extra bad since the load throughput used to be only 1/cycle. On the other hand, horizontal addition also used to be much worse than it is now. It just seems like all options are terrible. It depends on your audience of course, but frankly I don't think it's worth expending much energy on.


Bare bones implementation of Java HashMap I've done a minimal implementation of a HashMap. Invite comments.

public class HashMap<K, V> {

    private final int size = 16;
    private Node<K, V>[] arr;

    public HashMap(){

        arr = new Node[size];
    }

    class Node<K, V> {

        private int index;
        private K key;
        private V value;
        private Node<K, V> next;

        protected Node(K key, V value){
            this.key = key;
            this.value = value;
        }

    }

    private int getHashCodeForKey(K key){
        int result = 7;
        return 31 * result + (key != null ? key.hashCode() : 0);
    }

    public V get(K key){

        int hashcode = getHashCodeForKey(key);
        int index = hashcode & size;
        Node head = arr[index];
        while(head != null){
            if(head.key == key) return (V) head.value;
            head = head.next;
        }
        return null;

    }

    public void put(K key, V value){
        Node curr = new Node(key, value);
        int hashcode = getHashCodeForKey(key);
        int index = hashcode & size;

        if(arr[index] == null){
            arr[index] = curr;
        }else{
            Node head = arr[index];
            curr.next = head;
        }
    }

}

Strange constant usage
This code seems strange to me:

private int getHashCodeForKey(K key){
    int result = 7;
    return 31 * result + (key != null ? key.hashCode() : 0);
}

Why set result to 7, just to multiply it by 31?  That is, why not this:
private int getHashCodeForKey(K key){
    int result = 7 * 31;
    return result + (key != null ? key.hashCode() : 0);
}

It makes me think that you wanted to multiply the whole thing by 31 but I'm not sure of your intent.
Weakening the hash
The code that uses the hash code:

    int hashcode = getHashCodeForKey(key);
    int index = hashcode & size;
    Node head = arr[index];

weakens the hash because you are using power of 2 sized arrays (in your case a fixed size 16 array).  So in your case you are using only 4 bits of the hash.  If your array size were 17 and you used % 17, it would use all the bits of the hash.
Poor performance
With a fixed size of 16 buckets, your hash will quickly devolve into a linear search once you add a lot of entries.  A proper hash map would resize itself at appropriate intervals to maintain performance.
Correctness
This comparison is not correct:

if(head.key == key)

For example, two strings that have the same contents will not compare correctly using the above code.  You should change the comparison to:
if (head.key.equals(key))


Connect to streaming Twitter API, parse tweets, and write to CSV in real-time I have been using Tweeepy to connect to the Twitter Streaming API to collect tweets, parse that data, and then write select fields to a CSV file.  Based on some examples that I found, I put together the following code to manage that connection.  One thing that I had to work around was how to handle the connection getting killed.  I was able to include an while-loop that properly handles exceptions and restarts the connection if needed.

I want to make sure that this code is optimized and that I'm not including things that might not be needed.

#!/usr/bin/env python

import logging
import time
import csv
import json
import tweepy
from tweepy import OAuthHandler
from tweepy import Stream
from tweepy.streaming import StreamListener
from datetime import datetime
from dateutil import parser

# enable logging
logging.basicConfig(level=logging.INFO,
                    format='%(asctime)s %(levelname)s %(module)s - %(funcName)s: %(message)s',
                    datefmt="%Y-%m-%d %H:%M:%S")
logger = logging.getLogger()
logger.setLevel(logging.INFO)

# authorize the app to access Twitter on our behalf
consumer_key = ' '
consumer_secret = ' '
access_token = ' '
access_secret = ' '
auth = OAuthHandler(consumer_key, consumer_secret)
auth.set_access_token(access_token, access_secret)
api = tweepy.API(auth)

# establish open connection to streaming API
class MyListener(StreamListener):

    def on_data(self, data):
        try:
            tweet = parse_tweet(data)
            content = extract_content(tweet)
            with open('tweets.csv', 'a') as f:
                writer = csv.writer(f, quotechar = '"')
                writer.writerow(content)
                #logger.info(content[3])

        except BaseException as e:
            logger.warning(e)

        return True

    def on_error(self, status):
        logger.warning(status)
        return True

# parse data
def parse_tweet(data):

    # load JSON item into a dict
    tweet = json.loads(data)

    # check if tweet is valid
    if 'user' in tweet.keys():

        # parse date
        tweet['CREATED_AT'] = parser.parse(tweet['created_at'])

        # classify tweet type based on metadata
        if 'retweeted_status' in tweet:
            tweet['TWEET_TYPE'] = 'retweet'

        elif len(tweet['entities']['user_mentions']) > 0:
            tweet['TWEET_TYPE'] = 'mention'

        else:
            tweet['TWEET_TYPE'] = 'tweet'

        return tweet

    else:
        logger.warning("Imcomplete tweet: %s", tweet)

# extract relevant data to write to CSV
def extract_content(tweet):

    content = [tweet['user']['screen_name'],
               tweet['CREATED_AT'].strftime('%Y-%m-%d %H:%M:%S'),
               tweet['TWEET_TYPE'],
               tweet['text'].encode('unicode_escape')]

    return content

def start_stream():

    while True:

        logger.warning("Twitter API Connection opened")

        try:
            twitter_stream = Stream(auth, MyListener())
            twitter_stream.filter(track=['nyc'])

        except Exception as e:
            logger.warning(e)
            continue

        finally:
            logger.warning("Twitter API Connection closed")

if __name__ == '__main__':
    start_stream()

MyListener is not a very descriptive name. Maybe CSVWriter would be a better one. And that name already suggests something else, that it should take the file name as a parameter:

class CSVWriter(StreamListener):
    def __init__(self, file_name, *args, **kwargs):
        self.file_name = file_name
        StreamListener.__init__(*args, **kwargs)

    def on_data(self, data):
        try:
            tweet = parse_tweet(data)
            content = extract_content(tweet)
            with open(self.file_name, 'a') as f:
                writer = csv.writer(f, quotechar = '"')
                writer.writerow(content)
        except Exception as e:
            logger.warning(e)
        return True

It is also enough to except Exception. BaseException will catch a bit more than you would probably want, namely KeyboardInterrupt (so you can press Ctrl-C to stop the program).
        ...

twitter_stream = Stream(auth, CSVWriter("tweets.py"))

There seems to be no need to first save tweet['CREATED_AT'] = parser.parse(tweet['created_at']) and then later do tweet['CREATED_AT'].strftime('%Y-%m-%d %H:%M:%S'). Just do the correct conversion at the end (I don't know which one exactly, because I don't know what format it is in, orginally).

I would also reduce the number of blank lines. While some are nice for readability (to separate blocks of code) or are recommended by PEP8, blank lines between before elif or else block just break up the reading and reduce readability.


XHTML input for PHPmailer I have used PHPmailer to send email to users upon registering to the site. Now, everything works as should, however, I find myself doing painstaking work adding the HTML elements in.

Here's an example piece of HTML code:

This is a paragraph

 Now this is important!

Uh Oh!

Now in the PHP class, I would have to '' and . after each HTML element, so for example:

'
'.
'
'.
''.
'
'.
'
';

As you could imagine, doing this for a whole XHTML page, can take a very long time. I was wondering if there's a method that's quicker and more efficient than my technique.

Here's my entire PHP class that includes the HTML elements:

        $mail->Body =
        '<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">'.
        ''.
        ''.
        '  <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />'.
        '  <title>Demystifying Email Design</title>'.
        ' <link href="https://fonts.googleapis.com/css?family=Dosis" rel="stylesheet" type="text/css"/>'.
        '  <meta name="viewport" content="width=device-width, initial-scale=1.0"/>'.
        ''.
        ''.
        '
'.
        '
'.
        ''.
        '
'.
        '
'.
        '
      '.
      '	'.
      '
'.
      '
'.
      ''.
      '
'.
      '
'.
      '';
      echo "Hello there hero!";
      '	'.
      '
'.
      '
'.
      '';
      echo " Cheesy heading, we know. BUT! Thanks for signing up with us! We just need you to do one, tiny little thing! You see that button down there? The big yellow one? We're going to need you to click that.. Seems daunting, but trust us. Once you do, you'll get access to all our websites features! Easy right? So go ahead, what are you waiting for?";
      '	'.
      '
'.
      '
'.
      ''.
      '
'.
      '
'.
      ''.
      '
'.
      ''.
      '
';
      echo "Verify Account!";
      '
'.
      '	'.
      '
'.
      '
';

You may use a template.

$message = file_get_contents('path/to/your/html/template.html');

Then replace parts of it applying variables

// Defining variables
$variable1 = 'This will be a heading';
$variable2 = 'This will be a paragraph. Look at me, I\'m awesome!';

// Replace placeholders with my variables
$message = str_replace('%variable1%', $variable1, $message);
$message = str_replace('%variable2%', $variable2, $message);

And finally pass it to PHPmailer

$mail->Body = $message;

In your HTML template reference variables as part of your body, nothing more than

%variable1%

%variable2%

You probably already know, but I'd suggest anyway to specify that the message contains HTML

$mail->IsHTML(true);         // specify that it is html
$mail->CharSet = "utf-8";    // avoid encoding issues


Scanning directories and caching paths to an array What am I trying to do

I'm trying to scan a disk and save (cache) all the paths of files and directories into an array so that I could perform a quick file/directory search later on.

Implementation

I'm using a Node.js module called walkdir.

cacheDisk () {
  let searchDir = 'C:/'
  let cache = []

  let options = {
    "max_depth": 5,       // only recurse down to max_depth. if you need more than no_recurse
    "track_inodes": true  // should be used with max_depth to prevent infinite loop. On windows or with hardlinks some files are not emitted due to inode collision.
  }

  let paths = walkdir(searchDir, options, (path) => {
    cache.push(path)
  })
  .on('end', () => {
    console.log(cache)
    this.findItem(cache)
  })
},

findItem (cache) {
  let filename = 'file_1.txt'
  const matches = cache.filter(element => element.includes(filename))
  console.log(matches)
}

Questions / Problems:

FYI: I only need it to work in the latest Chrome (it's an Electron app)

Is using cache.push(path) a good way (performance-wise) to save paths to the array or is the module already doing it for me and I'm just repeating this action on every iteration? It takes about 30 seconds and 500-800 MB of RAM on a core i7 to finish that caching operation for my ssd with 1 million paths

Is using cache.filter(element => element.includes(filename)) a good / safe way to find an element in an array that can potentially be ~10,000,000 paths in length?

When I'm trying to scan disk's root (e.g. C:/) I get 2 slashes after the disk letter in every path (e.g. C:\\Program Files\test). But when I'm scanning something like C:/test/ everything is fine. I guess that's Node's doing? How do I avoid that? (do I just replace all \\ with \ in all the paths afterwards?)

I have to ask why you would want to find a file anywhere on the filesystem. That seems like a pretty odd requirement. Most apps (take node module loading for example) check a limited number of folders which can be done very quickly.

To answer your questions:

the OS does not do this kind of caching.
No, I wouldn't consider this safe. For two reasons. There are many other names that could match. For example video_file_1.txt, file_1.txt.old or some_path/file_1.txt/cat.jpg.  I would check against the basename something like

function basename(path) {
   return path.match(/[^\\\/]+$/)[0];
}
const matches = cache.filter(element => basename(element) == filename)

Someone already answered this in the comments. It is really just an artifact of attempting to handle windows and unix paths consistently.

Also if retrieval efficiency is important then I would store the data as a hash not an array, i.e:

let cache = {};

let paths = walkdir(searchDir, options, (path) => {
  const name = basename(path);
  if (!cache[basename])
    cache[basename] = [];
  cache[basename].push(path)
});

Then finding an element is a simple matter of:

findItem (cache) {
  let filename = 'file_1.txt';
  const matches = cache[filename];
  console.log(matches);
}

Note that, since you are on Windows, you should also consider case-insensitivity of filenames.