How to make this code follow Solid Principle C# Currently I have this code:
private async Task<(List<SomeModel> data1, List<string> data2)>
                    ProcessFileAsync(IEnumerable<FileInfo> filesInDirectory)
{
    var data1 = new List<SomeModel>();
    var data2 = new List<string>();

    var inDirectory = filesInDirectory as FileInfo[] ?? filesInDirectory.ToArray();
    for (var lIndex = 0; lIndex < inDirectory.Count(); lIndex++)
    {
        var content = await File.ReadAllTextAsync(inDirectory.ElementAt(lIndex).FullName);

        var name = Path.GetFileNameWithoutExtension(inDirectory.ElementAt(lIndex).Name);

        var data1Task = GetData1Async(content, name);
        var data2Task = GetData2Async(content);

        await Task.WhenAll(data1Task, data2Task);

        data1.AddRange(data1Task.Result);
        data2.AddRange(data2Task.Result);
    }

    return (data1, data2);
}

The calling code:
private async Task<(List<SomeModel> data1, List<string> data2)> BatchProcessFilesAsync()
{
    var batchSize = 4;

    var models
        = await GetDataFromSomewhereAsync().ConfigureAwait(false);
    var listOfRequests = new List<Task<(List<SomeModel> data1, List<string> data2)>>();

    var fileModels = models.ToList();
    for (var skip = 0; skip <= fileModels.Count(); skip += batchSize)
    {
        var files = fileModels.Skip(skip).Take(batchSize);
        listOfRequests.Add(ProcessFileAsync(files));
    }

    // This will run all the calls in parallel to gain some performance
    var allFinishedTasks = await
    Task.WhenAll(listOfRequests).ConfigureAwait(false);

The problem I am facing is if I need to add data3 in future, i have to add code here again. Is there a way to make this code Solid.

Let me show you how to review your code from SOLID principles point of view.
Here I try to list some general questions and give some guidance to help you determine that your current solution does violate some of the principles or not.
Single - Responsibility principle
Q: What does XYZ method do?
If your answer resembles to any of these:

First it ... then it ...
It does ... and it ...
It calls ... method and ... method
etc.

it is a good sign that your code method does too much things.
Just to make it clear this principle does NOT suggest to wrap each line into a separate function. On the contrary, it suggests to combine those that are closely related to each other to enhance cohesion. We should strive/aim for high cohesion and low coupling.
Open - Closed principle
Q: Is your XYZ method suitable for T or W type of changes?
In this principle you try to guess what will future present you. Let's suppose that XYZ can handle X, Y and Z. If you need to support T and W then do you need to modify the current code or can you for example inherit from that and override only the necessary methods?
Liskov Substitution principle
Not applicable here.
Interface Segregation principle
Q: Do I (as a consumer of your API) have to know what does each method of IXYZ do?
This principle suggests to try to think from the consumer perspective. If I want to call only X then do I need an interface which provides X, Y and Z? You can think of this principle as an extension of SRP to interfaces. If your interface contains members that are not highly coherent then you need to slip it.
Dependency Inversion principle
Q: Does your XYZ class / method depend on other concrete classes?
This principle suggests to depend on abstractions rather than on concrete types. (Remember we are aiming for loose coupling). If you depend on abstraction and not the implementation then you can switch easily without the need to modify this class (this helps OCP as well).

I hope this helped you a bit. I suggest to further read these excellent articles:

What to look for in a Code Review: SOLID Principles
The Zen of Code Reviews: Review As If You Own the Code
What to look for when reviewing code for SOLID principle ‘violations’


Solution to basic unit tests I'm working my way through an intermediate Python course on Pluralsight that focuses on unit testing and just finished my first assignment.

Basically the course itself focuses on teaching you how to write correct test-cases and gives you a baseline of the class to start solving the tests by yourself.

To clarify:

I'm more concerned on the implemented solutions to the test-cases than on the test-cases themselves. Sorry for any confusion!

I was hoping that I could get some feedback on my implemented solutions to the test-cases (they all are passing):

import unittest

from phonebook import Phonebook

class PhoneBookTest(unittest.TestCase):

    def setUp(self):
        self.phonebook = Phonebook()

    def test_phonebook_with_normal_entries_is_consitent(self):
        self.phonebook.add("Bob", "12345")
        self.phonebook.add("Mary", "012345")
        self.assertTrue(self.phonebook.is_consistent())

    def test_phonebook_with_duplicate_entries_is_inconsitent(self):
        self.phonebook.add("Bob", "12345")
        self.phonebook.add("Sue", "12345")  # identical to Bob
        self.assertFalse(self.phonebook.is_consistent())

    def test_phonebook_with_same_prefix_is_inconsistent(self):
        self.phonebook.add("Bob", "12345")
        self.phonebook.add("Sue", "123")  # prefix of Bob
        self.assertFalse(self.phonebook.is_consistent())

Here is my actual Phonebook class:

class Phonebook:

    def __init__(self):
        self._entries = {}

    def add(self, name, number):
        self._entries[name] = number

    def lookup(self, name):
        return self._entries[name]

    def is_consistent(self):
        if not self._entries:
            return True
        else:
            if self.check_duplicate():
                if self.check_prefix():
                    return True
        return False

    def check_duplicate(self):
        seen = set()
        for x in self.get_values():
            if x not in seen:
                seen.add(x)
            else:
                return False
        return True

    def check_prefix(self):
        prefixes = []
        for contact_number in self.get_values():
            if contact_number not in map(lambda number: number[0:len(contact_number)], prefixes):
                prefixes.append(contact_number)
            else:
                return False
        return True

    def get_values(self):
        return self._entries.values()

I'm mostly concerned of refactoring and the check_function and check_prefix functions performance and overall implementation. However I still consider myself to be a beginner, so all help would be appreciated! :)

Any feedback would be great!

Bug: The result of check_prefix depends on the order of values. Basically, it compares each pair of values once, but truncates only one of the two for comparison. On my Python 3.5 your test fails occasionally.
Duplicate values are also prefixes of one another. It is not necessary for is_consistent to check for both.
It is not necessary to handle the empty phonebook as a special case in is_consistent as the check functions return True swiftly anyway in that case. Thus the function can be simplified to

def is_consistent(self):
    return self.check_prefix()

A more efficient solution for check_prefix would be to sort the values and compare adjacent ones. Making use of pairwise from Itertool recipes, and the fact that the prefix will sort first:

def check_prefix(self):
    return not any(b.startswith(a)
                   for a, b in pairwise(sorted(self.get_values())))


Berlekamp–Massey Algorithm in Haskell I was recently tasked to write an implementation of the Berlekamp–Massey algorithm (for GF2) from a given inductional proof and decided to fresh up on my Haskell, since converting a proof by induction into a recursive algorithm should be quite straight-forward.

The variable names have been taken from the given proof (and thus can't be changed, although they differ from other notations or are rather short); here's a map:

ks: the input bit stream
len: the length of ks
n: degree of the current annihilating polynomial
phi: said polynomial, where a "bit" set at position \$i\$ indicates that the the \$i\$-th power is "part" of the polynomial, ie.:
$$
      \varphi(X) = \sum_{i=0}^n \text{phi}[i] \cdot X^i
  $$
s, psi: analogously the degree and the polynomial itself of the last iteration that had different values
r: the maximum length of the prefix of ks that psi annihilates successfully
dl: the discrepancy between the annihilation by phi and the \$l\$-prefix of ks
eta: The new polynomial, calculated as:
$$
      \eta(X) = \varphi(X) + X^{l-r} \cdot \psi(X)
  $$

data State = State { n :: Int
                   , phi :: [Int]
                   , s :: Int
                   , psi :: [Int]
                   , r :: Int
                   }
    deriving Show

discr :: Int -> [Int] -> Int -> [Int] -> Int
discr l ks n phi = foldr (\ a k -> (a+k) `mod` 2) 0
    (zipWith (*) (take (n+1) phi) (reverse $ take (n+1) $ drop (l-n) ks))

buildEta :: [Int] -> [Int] -> Int -> Int -> Int -> [Int]
buildEta phi psi l r s
    = (take (l-r) phi) ++
      (zipWith (\ c b -> (c+b) `mod` 2) (take (s+1) $ drop (l-r) phi) (take (s+1) psi)) ++
      (drop (l-r+s+1) phi)

bk' :: Int -> [Int] -> State
bk' (-1) ks = State 0 (1:lenzeroes) 0 (1:lenzeroes) (-1)
    where lenzeroes = replicate (length ks) 0
bk' l ks
    | dl == 0   = State n phi s psi r
    | n*2 <= l  = State (l+1-n) eta n phi l
    | otherwise = State n eta s psi r
    where State n phi s psi r = bk' (l-1) (ks)
          dl = discr l ks n phi
          eta = buildEta phi psi l r s

bk :: [Int] -> (Int,[Int])
bk ks = (n,phi)
    where State n phi _ _ _ = bk' (length ks) ks

I'm not quite happy with the result though, especially the buildEta function and the use of length.  I think the latter could be improved by reversing ks and then giving only the tail to the recursive call, but somehow I still need to build the phi and psi lists correctly, which I fail to do "nicely".

Style aside, the code works (as far as I know), eg.:

*Main> bk [1, 0, 1, 0, 0, 1, 1, 0, 1]
(5,[1,0,1,0,1,1,0,0,0])

Which means that the degree of the polynomial is \$n\$ and it has the following form:
$$
\varphi = X^0 + X^2 + X^4 + X^5
$$
Or, as an LSFR, A=[1,1,0,1,0] which indeed generates ks with [1, 0, 1, 0, 0] as initialization sequence.

Edit: Had someone review my code offline and did some minor style fixes already:

some point-free style where appropriate
replacing some lambda with just (*)
replacing sum ()mod2 with foldr and lambda
found a way to rewrite the if-then-else into nice guards
replaced the list-comprehension for zeroes with replicate
line breaks :-)

A central question I ask myself on these is whether the explicit recursion follows an abstractable pattern. In this case, only bk' has explicit recursion, and its pattern is simple - the call tree is always linear and brings l incrementally down to -1.

buildEta applies a transformation to a particular slice of phi.

Why are you working with Ints? These are Bools.

bk :: [Bool] -> (Int,[Bool])
bk ks = (n,phi) where
  State n phi _ _ _ = foldr (bk' ks) (State 0 lenzeroes 0 lenzeroes (-1)) [l,l-1..0]
  lenzeroes = True : replicate l False
  l = length ks

bk' :: [Bool] -> Int -> State -> State
bk' ks l (State n phi s psi r)
  | not dl    = State n phi s psi r
  | n*2 <= l  = State (l+1-n) eta n phi l
  | otherwise = State n eta s psi r
  where dl = discr l ks n phi
        eta = overSlice (l-r) (s+1) (zipWith xor psi) phi

discr :: Int -> [Bool] -> Int -> [Bool] -> Bool
discr l ks n = foldr xor False . zipWith (&&) (reverse $ take (n+1) $ drop (l-n) ks)

overSlice :: Int -> Int -> ([a] -> [a]) -> ([a] -> [a])
overSlice from size f a = b ++ f d ++ e where
  (b, c) = splitAt from a
  (d, e) = splitAt size c


Finding out the prime factors of a number This finds the prime factors of a number given in a variable. Is it efficient or in need of improvement?

public class PrimeFactorFinder {
    public static void main(String args[]){
        long n = 18;
        long md = 2;
        while(n!=java.lang.Math.sqrt(n)){

            if(n%md==0){
                System.out.print(" "+md);
                n /= md;
            } else {
                md++;
            }
        }
    }
}

The efficiency can be improved. As Simon already said, the condition

n != java.lang.Math.sqrt(n)

is equivalent to n != 1, and therefore not of much use.
Perhaps you meant something different: A composite number \$ n > 1 \$
must have a prime factor \$ p \$ for which  \$ p \le \sqrt n \$. Therefore,
as soon as md > java.lang.Math.sqrt(n) in your iteration,
n is either equal to 1 or a prime number.

The code then is

    long md = 2;
    while (md <= java.lang.Math.sqrt(n)) {
        if (n % md == 0) {
            System.out.print(" " + md);
            n /= md;
        } else {
            md++;
        }
    }
    if (n > 1) {
        System.out.print(" " + n);
    }

which reduces the number of iterations considerably:
For n = 64_548_621 the loop runs until md = 283 instead of
md = 25343.

Another small improvement is to repeatedly test a possible factor
in a while-loop, in order to save some square root computations
of the (updated) number n:

    long md = 2;
    while (md <= java.lang.Math.sqrt(n)) {
        while (n % md == 0) {
            System.out.print(" " + md);
            n /= md;
        }
        md++;
    }
    if (n > 1) {
        System.out.print(" " + n);
    }

A simple measure to reduce the number of iterations to about 50% is to
test the factor 2 separately, and then only odd numbers starting
at 3:

    while (n % 2 == 0) {
        System.out.print(" " + 2);
        n /= 2;
    }
    long md = 3;
    while (md <= java.lang.Math.sqrt(n)) {
        while (n % md == 0) {
            System.out.print(" " + md);
            n /= md;
        }
        md += 2;
    }
    if (n > 1) {
        System.out.print(" " + n);
    }

As pointed out by @CodesInChaos in a comment,
computing the square root of a long integer via floating point
arithmetic can suffer from rounding issues.
This would be an issue for numbers greater than 2^52.
From the analysis at How can you easily calculate the square root of an unsigned long long in C?
it follows that the difference between
(long)java.lang.Math.sqrt(n) and the correct
value \$ \lfloor \sqrt n \rfloor \$ is at most one.
Since only an upper bound is needed, replace the loop condition by

    while (md <= java.lang.Math.sqrt(n) + 1) {

If you want  reusable code then move the computation of
the prime factors in a separate function which returns the list
of primes. This also separates the computation from the I/O
and keeps the main function short:

import java.util.List;
import java.util.ArrayList;
import java.util.Arrays;

public class PrimeFactorFinder {

    public static void main(String args[]){
        long n = 64_548_621;
        List<Long> factors = primeFactors(n);
        System.out.println(Arrays.toString(factors.toArray()));
    }

    static List<Long> primeFactors(long n) {
        List<Long> factors = new ArrayList<Long>();
        long md = 2;
        while (n % md == 0) {
            factors.add(md);
            n /= md;
        }
        md = 3;
        while (md <= java.lang.Math.sqrt(n) + 1) {
            while (n % md == 0) {
                factors.add(md);
                n /= md;
            }
            md += 2;
        }
        if (n > 1) {
            factors.add(n);
        }
        return factors;
    }
}


Regex to extract selective lines I have a list of error messages:

def errorMessages = ["Line : 1 Invoice does not foot Reported"
                     "Line : 2 Could not parse INVOICE_DATE value"
                     "Line 3 : Could not parse ADJUSTMENT_AMOUNT value"
                     "Line 4 : MATH ERROR"
                     "cl_id is a required field"
                     "File Error : The file does not contain delimiters"
                     "lf_name is a required field"]

I am trying to create a new list which doesn't match the regex "^Line\\s(?:(\\d+)\\s)?\\s*:\\s+(\\d+)?.+" but has the text Invoice does not foot Reported.

My desired new list should be like this:

def headErrors= ["Line : 1 Invoice does not foot Reported"
                 "cl_id is a required field"
                 "File Error : The file does not contain delimiters"
                 "lf_name is a required field"]

This is what I am doing for now:

regex = "^Line\\s(?:(\\d+)\\s)?\\s*:\\s+(\\d+)?.+"
errorMessages.each{
    if(it.contains('Invoice does not foot Reported'))
        headErrors.add(it)
    else if(!it.matches(regex)
        headErrors.add(it)
}

Is there a way it can be done just using regex instead of if else?

Use findAll() instead of each(). findAll() is a filter function which can be used on a Collection/Iterable or Map. It filters the result on satisfying the condition provided by the Closure parameter.

Here, you can filter String which has the desired text or does not match with the supplied regex. Using findAll is better than each because it returns the filtered collection as a result as compared to each which returns the Collection itself.

def regex = "^Line\\s(?:(\\d+)\\s)?\\s*:\\s+(\\d+)?.+"

def headErrors = errorMessages.findAll {
    it.contains('Invoice does not foot Reported') || !(it ==~ regex)
}

assert headErrors ==  [
    'Line : 1 Invoice does not foot Reported',
    'cl_id is a required field',
    'File Error : The file does not contain delimiters',
    'lf_name is a required field'
]


Breaking a chess coordinate string into 2D coordinates Given a chess coordinate as a string (e.g. "a1") I'd like to transform it into a 2-D array (so, for "a1" I'd like to get [1,1]).

Here's what I came up with:

def safe_pawns(pawns)
  pawns.inject([]){|res, pwn| res << [pwn.split('')[0].index(/[a-h]/) + 1, pwn.split('')[1].to_i]}
end

Can anyone suggest a refactoring please to make it more idiomatic Ruby?

Give your code some breathing room. I.e. ) { |a, b| instead of ){|a, b|. And there's no need to put everything on one line. The way your block works right now, it'd be better to store the result of pwn.split in a variable, instead of calling split twice. (There's also no need to abbrevate "pawn" as "pwn".)
That being said, strings support array-like access, so you don't need the split at all.
Judging from your code, pawns is an array. Transforming an n-element array to a new n-element array is called "mapping". You're using inject which is also known as reduce (and fold in many other languages; see comments) - an operation most often used to take an n-element array and reduce it to a single value. So step 1: Use map instead of inject.
I'd probably use a regex to pull the string apart. It'll double as a way to check the coordinate strings for validity (e.g. so no "z9" coordinates will slip through).
It's a little low-level, but we can use the fact that "a" is 97 in ASCII. So to get the number for a letter, we can say letter.ord - 96.

You get something like this:

def safe_pawns(squares)
  squares.map do |square|
    [$1.ord - 96, $2.to_i] if square.downcase =~ /^([a-h])([1-8])$/
  end.compact
end

Alternatively, if you're sure that all the input coordinates are valid, lowercase strings already, you don't need the regex or the downcasing:

def safe_pawns(squares)
  squares.map { |square| [square[0].ord - 96, square[1].to_i] }
end

As tokland points out in the comments, we can avoid the hardcoded 96 (which isn't very self-explanatory) and instead get the letter-to-number translation by saying:

square[0].ord - 'a'.ord + 1


Leaving out the max value in an Enumeration This is part of a school assignment and I came up with the following solution to leave out the max value in a Enumeration:

static void Opdracht3()
{
    List<int> values = new List<int>();
    foreach (int value in Enum.GetValues(typeof(Opleidingen)))
    {
        values.Add(value);
    }

    foreach (int value in values)
    {
        if (value != values.Max())
        {
            Console.WriteLine("Naam: {0}, Waarde: {1}", Enum.GetName(typeof(Opleidingen), value), value);
        }
    }
}

How could I improve this piece of (working) code?

This is not efficient:

foreach (int value in values)
{
    if (value != values.Max())

Because in every iteration, you recalculate values.Max(), when it would be enough to do it once, before starting the loop.

Also, you don't need to create values as a List<int>. As a result, the method can be simplified:

static void Opdracht3()
{
    int maxValue = Enum.GetValues(typeof(Opleidingen)).Cast<int>().Max();

    foreach (int value in Enum.GetValues(typeof(Opleidingen)))
    {
        if (value != maxValue)
        {
            Console.WriteLine("Naam: {0}, Waarde: {1}", Enum.GetName(typeof(Opleidingen), value), value);
        }
    }
}

And as @blueraja pointed out, to reduce duplication even further, it's good to cache Enum.GetValues(typeof(Opleidingen)).Cast<int>() in a variable:

static void Opdracht3()
{
    int[] values = Enum.GetValues(typeof(Opleidingen)).Cast<int>();
    int maxValue = values.Max();

    foreach (int value in values)
    {
        if (value != maxValue)
        {
            Console.WriteLine("Naam: {0}, Waarde: {1}", Enum.GetName(typeof(Opleidingen), value), value);
        }
    }
}


Using BigDecimal to validate epoch timestamp in exponential notation I have some phone records in which epoch time is coming in exponential format like 1467.738871E9. I need to validate those records in which number is not in proper format and drop those records. Right now, I am using big decimal, but I have heard that bigdecimal is slow and takes more space. Since I have million of records to be validated, is there any other way to validate time which is more space- and time-efficient?

try{
    new BigDecimal(data).toBigInteger();}
    catch(NumberFormatException n){
      return false;
    }
return true;

Examples of valid timestamps:

1.467983982E9
1.468119771E9
1467858808E9
1.468119771E11
1468119771E11

Examples of invalid timestamps:

1.46A802215E9
null
4.1B46A802215E9

There's no point in calling toBigInteger since that doesn't throw anything. You might as well just have this:

try {
    new BigDecimal(data);
} catch(NumberFormatException n) {
    return false;
}
return true;

Or you might consider using toBigIntegerExact if you don't allow fractional parts in the number (e.g. 1.2345E3 which is 1234.5 is not allowed):

try {
    new BigDecimal(data).toBigIntegerExact();
} catch(NumberFormatException | ArithmeticException e) {
    return false;
}
return true;

Also, have you thought about whether negative numbers should be allowed?

Regarding time & space performance, it will create one BigDecimal object (and possibly one BigInteger object) for every valid record. It will then release it as eligible for garbage collection. The JIT compiler will probably optimize this to make it insignificant compared to the time it takes to read and write the records.


Loading the resource identified by a script element's src attribute I have a function to load the resource identified by a <script> element's src attribute.

I need it to:

Be as universal as possible.
Work on file-to-file requests (from file:// to file://), if the browser allows to.

This is the code I have so far:

var asyncGetContents = function(node,listener){
   var hasRun = false;
   if(node.src){
      var httpRequest = new XMLHttpRequest();
      httpRequest.onreadystatechange = function() {
         if(httpRequest.responseText){
            if(hasRun){return;}
            hasRun = true;
            listener(httpRequest.responseText);
            httpRequest.onreadystatechange=undefined;
         }
      };
      httpRequest.open('GET', node.src);
      if(node.type){
         httpRequest.overrideMimeType(node.type);
      }
      httpRequest.send();
   }else{
      listener(node.textContent);
   }
};

Is this code well-formed?
Are there any aspects that I could've possibly missed?
Is there any way to do it without the variable hasRun? Removing the onReadyStateChange event listener doesn't seem to work - listener function runs twice.

The code looks well-formed. I'd like some comments though and maybe a bit of whitespace here and there.

However, I think there's a good chance this will actually fail. readystatechange fires several times during the lifetime of an XHR object. But your code doesn't actually check the XHR object's readyState, which is what the event is there to tell you about.

Instead it checks responseText, but if the file is large enough to fire several events during its download, responseText may only hold partial data. So responseText is truthy, and hasRun is set to true, but the code won't ever get the complete data.

There's are 5 distinct readyState values. You should be checking for the DONE state only.

Alternatively, for some (most, I believe) browsers you can simply hook your functions to onload, onerror and friends instead of of the general onreadystatechange.

You may also want to check the arguments: Does node exist? Is listener a function?

Lastly, how do you handle failures? A 404 will still give you responseText, just not anything useful. A simple solution would be something like Node.js's callback pattern, of having the listener receive 2 arguments: err and data. If err is truthy, something went wrong. You also want to simply pass the XHR object along as the 3rd argument, so the listener can make whatever checks it wants to (much like jQuery does).

function asyncGetContents(node, listener) {
  var xhr;

  // Example of handling a bad node argument
  // Adapt it to fit your needs
  if(!node) {
    throw new TypeError;
  }

  // Example of handling a bad listener argument
  // Adapt it to fit your needs
  if(typeof listener !== 'function') {
    throw new TypeError;
  }

  if(node.src) {
    xhr = new XMLHttpRequest();
    xhr.onreadystatechange = function () {
      // check readyState
      if(xhr.readyState === XMLHttpRequest.DONE) {
        // check that the response code is 2xx
        if(xhr.status >= 200 && xhr.status < 300) {
          listener(null, xhr.responseText, xhr);
        } else {
          listener(xhr.status, null, xhr); // Provide a more useful error here
        }
      }
    };

    xhr.open('GET', node.src);
    if(node.type) {
      xhr.overrideMimeType(node.type);
    }
    xhr.send();
  } else {
    // we've only checked that node is truthy, not that it's
    // actually a script element, so node.textContent may
    // be `undefined`, but let's leave that to the listener
    listener(null, node.textContent, null);
  }
}


Linq sorting on sort order of inner group Is this the best way to do this sort?

assets.OrderBy(a =>
    a.GalleryItems != null &&
    a.GalleryItems.FirstOrDefault() != null
        ? a.GalleryItems.OrderBy(gi => gi.SortOrder).First().SortOrder
        : 1000)

Basically I have a group of assets, which contain a group of gallery items with a sort order.  I need to get the lowest sort order from this group and sort the assets by that.  Gallery items can be null (it is coming through a service so I am unable to change this).

What about something like this:

assets.OrderBy(a => a.GalleryItems != null &&
                    a.GalleryItems.Any()
                    ? a.GalleryItems.Min(gi => gi.SortOrder)
                    : 1000)

It's actually a bit faster, since sorting the inner list takes longer than just getting the minimum value.


Slow SQL Server performance with multiple PreparedStatement loop I have a need to update thousands of database records at a time. I am using a loop to generate the necessary PreparedStatement updates, but am hoping for a more efficient method.

Normally, I would try to use a batch operation, but since the number of fields I need to update for each record differs, I need a new PreparedStatement for each update.

As it is, updates around 2,000 records takes 3-4 minutes.

            for (AgentUpdate update : updateList) {

                Agent thisAgent = update.getAgent();

                // Prefix for updating each agent
                StringBuilder sql = new StringBuilder(
                        "UPDATE REP_ASSIST.AGENTS SET\n"
                );

                // Create a map of all changes
                Map<String, String> updateMap = new HashMap<>();
                for (AgentDiffs diff : update.getChanges()) {
                    updateMap.put(diff.getAgentField().toString(), diff.getNewValue());
                }

                // Iterate through the map and build the SQL statement,
                // including all fields to be updated
                Iterator iterator = updateMap.entrySet().iterator();
                while (iterator.hasNext()) {
                    Map.Entry pair = (Map.Entry) iterator.next();
                    sql.append(pair.getKey()).append("=?");

                    if (iterator.hasNext()) {
                        sql.append(",\n");
                    }
                }
                sql.append("\nWHERE AGENT_CODE=? AND UPN=?;");
                Utility.printTempMsg(sql.toString());

                // Create the PreparedStatement and fill each ? with the
                // appropriate new value.
                ps = connection.prepareStatement(sql.toString());
                int paramNum = 1;
                for (String fieldName : updateMap.keySet()) {
                    String newValue = updateMap.get(fieldName);
                    ps.setString(paramNum, newValue);

                    // Increase the parameter count
                    paramNum++;
                }

                // We are now to our last two parameters, fill them
                ps.setString(paramNum++, thisAgent.getAgentCode());
                ps.setInt(paramNum, thisAgent.getAgentUpn());

                ps.executeUpdate();
                count++;

                Utility.printTempMsg(sql.toString());

                // Update the progress based on total count so far
                updateProgress(count, totalOperations);

            }

So how does one handle large dynamic updates of this sort?

Here are some ideas to try:

You are not closing your PreparedStatements.  There might possibly be some slow down due to resource consumption or deadlock.  Use the try-with-resources feature so it will automatically close.

for (AgentUpdate update : updateList) {
    try (PreparedStatement ps = connection.prepareStatement(sql.toString())
    {
        // set parameters
        ps.executeUpdate();
    }
}

Alternatively, you are creating a custom query and creating a new PreparedStatement for each record.  Although you are optimizing individual queries by minimizing the number of fields you are updating, you are creating additional overhead by preparing a new statements for each record.  If you update every field for every row, you could re-use the same PreparedStatement, which may end up being more efficient (depending on the number of fields and length of data you are actually updating).  Just overwrite the values in the prepared statement and redo the executeUpdate() on it.

try (PreparedStatement ps = connection.prepareStatement(sql.toString())) {
    for (AgentUpdate update : updateList) {
        // set parameters
        ps.executeUpdate();
    }
}


Compute text area given letter heights and a text I have solved the designer pdf question with haskell. The code works. I am posting the code here to know how I could have done it better.

  First line contains the weight of each alphabet
  Second line contains the word.

  Sample input

1 3 1 3 1 4 1 3 2 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 7
zaba

  Output

28

  Explanation
  each character would take 1 space and multiply it with the max weight
  4 characters * 1 space * 7 weight = 28
  'z' has the max weight of 7.

Code

import Data.List
import Data.Maybe

getintval::(Maybe Int) -> Int
getintval Nothing = 1
getintval (Just x) = x

solve'::[Char]->[(Char, Int)] -> Int
solve' ch lst = k $  map getintval $ map (\x -> finder' x) ch
  where
  k::[Int] -> Int
  k fb = (*) (length ch)   $ foldl (\acc x -> max acc x) 1 fb
  finder'::Char -> Maybe Int
  finder' g =  case i of
            Just(x1,x2) -> Just x2
            Nothing -> Just 1
            where i = find(\(val1, val2) -> val1 == g) lst

solve::[Char] -> [Char] -> Int
solve wght val = solve' val rec
  where
  rec::[(Char, Int)]
  rec = zipWith (\x y -> (x, y)) ['a'..'z'] word1
  word1::[Int]
  word1 = map(read::String->Int) $ words wght

main::IO()
main = do
  weight <- getLine
  pdfstr <- getLine
  putStr . show $ solve weight pdfstr

getintval is just fromMaybe 1. finder always produces Just - you probably meant to map Nothing to Nothing. finder' is just lookup. foldl (\acc x -> max acc x) 1 is just maximum (so long as ch is never empty). k can be inlined.

solve' :: [Char] -> [(Char, Int)] -> Int
solve' ch lst = (*length ch) $ maximum $ map (fromMaybe 1 . (`lookup` lst)) ch

zipWith (\x y -> (x, y)) is just zip. read's type can be deduced. rec and word1 can be inlined.

solve :: [Char] -> [Char] -> Int
solve wght val = solve' val $ zip ['a'..'z'] $ map read $ words wght

mapMaybe throws away invalid characters. We can flatten the call tree by letting main assemble the pieces.

solve :: [Char] -> [(Char, Int)] -> Int
solve ch lst = (*length ch) $ maximum $ mapMaybe (`lookup` lst) ch

parse :: [Char] -> [(Char, Int)]
parse = zip ['a'..'z'] . map read . words

main :: IO ()
main = do
  weight <- getLine
  pdfstr <- getLine
  print $ solve pdfstr $ parse weight


Multi-Threaded Cellular Automata Lab I am trying to improve the performance of my cellular automata lab. I have two arrays of Doubles representing the current values and the next values.

If I run the calculation in a single thread I get about 28 steps per second. However, if I break the work down into 2, 3 or 4 chunks and pass them into a concurrency queue I still get about 28 steps per second. If I increase the chunks further the algorithm takes longer and longer to complete, for example 10 chunks drops the performance to about 10 steps per second.

I'm testing this on a 3rd generation iPad Pro with 4 performance cores and 4 efficiency cores.

func step(from: Int, to: Int) {
    for j in from..<to {
        for i in 0..<w {
            AEMemoryClear(memory);

            AEMemorySetValue(memory, sI,                        cells[i   + (j  )*w])
            AEMemorySetValue(memory, aI, i != 0 && j != 0 ?     cells[i-1 + (j-1)*w] : 0)
            AEMemorySetValue(memory, bI, j != 0 ?               cells[i   + (j-1)*w] : 0)
            AEMemorySetValue(memory, cI, i != wB && j != 0 ?    cells[i+1 + (j-1)*w] : 0)
            AEMemorySetValue(memory, dI, i != wB ?              cells[i+1 + (j  )*w] : 0)
            AEMemorySetValue(memory, eI, i != wB && j != hB ?   cells[i+1 + (j+1)*w] : 0)
            AEMemorySetValue(memory, fI, j != hB ?              cells[i   + (j+1)*w] : 0)
            AEMemorySetValue(memory, gI, i != 0 && j != hB ?    cells[i-1 + (j+1)*w] : 0)
            AEMemorySetValue(memory, hI, i != 0 ?               cells[i-1 + (j  )*w] : 0)

            AERecipeExecute(recipe, memory)

            next[i + j*w] = memory.pointee.slots[index].obj.a.x
        }
    }
}
func step() {

    let start = DispatchTime.now()

    let n: Int = 4
    let z: Int = h/n

    let group = DispatchGroup()
    for i in 0..<n {
        group.enter()
        DispatchQueue.global(qos: .userInteractive).async { [unowned self] in
            self.step(from: i*z, to: i == n-1 ? self.h : (i+1)*z)
            group.leave()
        }
    }

    group.notify(queue: .main) { [unowned self] in
        (self.cells, self.next) = (self.next, self.cells)

        let end = DispatchTime.now()
        let delta = Double(end.uptimeNanoseconds - start.uptimeNanoseconds)/1000000000
        let target: Double = 1.0/60
        print("Time to calculate: \(delta) or \(round(1/delta)) SPS which is \(round(delta/target*100*10)/10)% of target; # of cells: \(self.w)^2 = \(self.w*self.h); seconds per cell: \(delta/Double(self.w*self.w))")
    }

    group.wait()
}

Also, another weird thing I'm noticing: if I run the calculation once a second it takes more than twice as long to complete than if I run it several times a second.  The only reason I can possibly think is that its using the efficiency core instead of the performance core in that case.

Note: AEMemorySetValue, AERecipeExecute, AEMemoryClear are C functions.

h and w are the cell dimensions of the cellular automata; the height and width. In practice they are the same and are about 300-500, depending on the device.  Also, h, w, index, sI, aI...hI are all static values that do not change at all through this entire process.

I also entirely moved the inner step function from Swift to C, but that had zero effect on the performance, positive or negative.

Before we comment on the speed issue, perhaps we should talk about the multithread correctness of step(from:to:). You are referencing memory and index. You appear to be updating the same memory from all of your threads. That is not thread-safe. Your don’t want multiple threads updating the same memory reference. If you temporarily turn on the thread sanitizer (known as TSAN, found in “Product” » “Scheme” » “Edit Scheme...” (or press ⌘+<) and then choose the “Diagnostics” tab), it may well warn you of this problem.

Let’s assume for a second that we get the thread-safety question behind us. The multithreaded performance concerns include:

Make sure you are striding to ensure that there is enough work on each thread, to ensure that the parallelism benefits are not outweighed by thread management overhead. That having been said, you are striding already with sufficient work on each thread, so that’s unlikely to be the problem here, but it often plagues naive parallelism attempts.
You may want to use concurrentPerform rather than just dispatching a bunch of blocks to a global queue and using a dispatch group to determine when they finished. The concurrentPerform will spin up the correct number of threads appropriate for your hardware, whereas if you write your own dispatching to concurrent queues, you might suboptimize the solution (e.g. you can exhaust the limited number of worker threads if you’re not careful).

In your case, the benefit of concurrentPerform will likely not be material, but it’s the first thing to consider when writing code that is effectively parallelized for loops. See https://stackoverflow.com/a/46499306/1187415 and https://stackoverflow.com/a/39949292/1271826 for introductions to concurrentPerform.
The key issue here is that one needs good algorithm design to minimize memory contention issues, cache sloshing, etc.

But let’s assume that you are really accessing/updating different index values through something not shared in your code snippet and are updating adjacent memory addresses. That can still result in problems. See below.

You said:

  I believe memory (and “recipe”) are going to need separate instances per “stride”. It's not clear to me exactly how that is slowing things down...

Yes, that’s likely the issue in this case. When writing multithreaded code, you have to be sensitive to the memory locations updated by parallel threads. For example, consider the following that adds the numbers between 0 and 100 million, updating an array of four values as it goes along, in parallel:

var array = Array(repeating: 0, count: 4)

DispatchQueue.global().async {
    DispatchQueue.concurrentPerform(iterations: 4) { index in
        for i in 0 ..< 100_000_000 {
            array[index] += i
        }
    }
    ...
}

Optimized for speed, that takes roughly 1.3 seconds on my machine (almost 3× slower than singled-threaded implementation). The issue in this contrived example is that we have multiple threads updating memory addresses very close to each other. You end up having CPU cores “sloshing” memory caches back and forth as the four threads are all trying to update the same block of memory.

For illustration purposes, consider the same code, except that instead of updating items right next to each other, I space them out by padding the array, grabbing the items at index values of 0, 1000, 2000, and 3000 (reducing CPU cache misses):

var array = Array(repeating: 0, count: 4 * 1000)

DispatchQueue.global().async {
    DispatchQueue.concurrentPerform(iterations: 4) { index in
        let updatedIndex = index * 1000
        for i in 0 ..< 100_000_000 {
            array[updatedIndex] += i
        }
    }
    ...
}

This is deliberately not touching much of the array, just updating four values. This looks absurd (and is not advisable) and one wouldn’t be faulted for assuming that this is far less efficient with all of that wasted memory. But it is actually 5× faster, taking roughly 0.25 seconds on the same machine.

That’s obviously an absurd example, but it illustrates the problem. But we can make it far more efficient like so:

var array = Array(repeating: 0, count: 4)

let synchronizationQueue = DispatchQueue(label: "sync")
DispatchQueue.global().async {
    DispatchQueue.concurrentPerform(iterations: 4) { index in
        var value = synchronizationQueue.sync { array[index] }
        for i in 0 ..< 100_000_000 {
            value += i
        }
        synchronizationQueue.async { array[index] = value }
    }
    ...
}

Even with the extra synchronization code to make sure that I have thread-safe interaction with this array, it’s still now another 5× faster, taking 0.05 seconds. If you can minimize updates to the same block of memory, you can have a material impact on performance, starting to enjoy the benefits of parallelism.

Bottom line, you have to be very careful about how multiple threads update shared memory blocks and balance workloads between the threads. It can have a considerable impact on performance.


Uploading a file asynchronously I've been experimenting with some old code that needs refactoring in places and was testing if there was any improvement to iis threads etc by uploading file asynchronously (Server Side). I'm using jQuery file upload client side.

The original code

[HttpPost]
public ActionResult UploadDocument( HttpPostedFileBase uploadedFile ) {

  // Do any validation here

  // Read bytes from http input stream into fileData
  Byte[] fileData;

  using ( BinaryReader binaryReader =
          new BinaryReader( uploadedFile.InputStream ) ) {

    fileData = binaryReader.ReadBytes( uploadedFile.ContentLength );

  }

  // Create a new Postgres bytea File Blob ** NOT Async **
  _fileService.CreateFile( fileData );

  return Json(
    new {
      ReturnStatus = "SUCCESS" // Or whatever
    }
  );

}

The new code

[HttpPost]
public async Task<ActionResult> UploadDocumentAsync( HttpPostedFileBase uploadedFile ) {

  // Do any validation here

  // Read bytes from http input stream into fileData
  Byte[] fileData = new Byte[uploadedFile.ContentLength];

  await uploadedFile.InputStream.ReadAsync( fileData, 0, uploadedFile.ContentLength );

  // Create a new Postgres bytea File Blob ** NOT Async **
  _fileService.CreateFile( fileData );

  return Json(
    new {
      ReturnStatus = "SUCCESS" // Or whatever
    }
  );

}

The new method appears to work correctly but my question is:

Is the following code the correct (best) way to do it? and are there any gotchas doing it this way? There is a lot of contradictory and out of date information out there. There also seems to be a lot of debate on whether there is any improvement or point in actually doing this. Yes it give back threads to iis etc but is it worth the overhead type of debate.

The code in question

// Read bytes from http input stream into fileData
Byte[] fileData = new Byte[uploadedFile.ContentLength];

await uploadedFile.InputStream.ReadAsync( fileData, 0, uploadedFile.ContentLength );

Your proposed changes will have marginal impact.  The way IIS works, IHttpHandlers are not invoked until the entire request body has been transferred.  That is, the entirety of the file data is already available by the time your first line of code runs.  That means your stream copying is just moving bytes around in memory, and while that is not instantaneous it's probably not slow enough that it's worth making it asynchronous.

I think you already know this, but you should make the Postgres call asynchronous if that's an option.  With the current call, you have a thread taken up waiting for a remote process to do its work.  With an asynchronous call, that thread would be available to do real work.  Of course, that only makes a difference if there is an incoming request that's waiting for a thread.


Program for an atoi() function in Java I have written this code to check all the edge cases in the implemention of an atoi() function.Please review.

//Program to print an atoi().

  public class Atoi{
      public static int atoiFunc(String str,char[] c,int l){

    //If the length is 0,the function returns 0;
    if(l==0){
        return 0;
    }

    //variable to store the result
    int res = 0;
    //variable to check for the sign.Initially,initially with positive 1.
    int sign = 1;

    // try/catch block to check the bounds of Integer datatype.
    try{
        int check = Integer.valueOf(str);
    }
    catch(NumberFormatException e){
        System.out.println("Number is going out of bounds of integer
               datatype.");
        System.exit(0);
    }

    for(int i=0; i<l; i++){
        if(c[i]=='-'){
            i++;
            sign = -1;
        }
        if(c[i]=='+'){
            i++;
        }
        if(c[i]!=' '){
            res = res*10 + c[i] -'0';
        }
    }
    return sign*res;
}

public static void main(String[] args) {

    String s = "-9846032";

    char[] ch = s.toCharArray();

    int len = s.length();

    int val = atoiFunc(s,ch,len);

    System.out.println("Original   : "+s);

    System.out.println("After atoi : "+val);

    }
}

String s = "-9846032";
char[] ch = s.toCharArray();
int len = s.length();
int val = atoiFunc(s,ch,len);

How come you chose to do it this way? In general, you shouldn't pass every permutation of a value into a function. If the function wants s.toCharArray, it can call it itself.

Imagine if we did the following:

String s = "-9846032";
char[] ch = "abcdef".toCharArray();
int len = "lololololololololol".length();
int val = atoiFunc(s,ch,len);

What does that even mean? To protect yourself from such nonsense and to make your API easier to understand, we should remove them. If you remove them, it means that your function can be called as such:

int val = atoiFunc("12345678")

which is much more preferable.

//If the length is 0,the function returns 0;
if(l==0){
    return 0;
}

What happens if l is -1? Or any other negative number? What happens if l is e.g. 12? Can you convert a 12 digit number to an int?

Importantly: How does someone differentiate atoiFunc("0") from atoiFunc("") ?

//variable to store the result
int res = 0;
//variable to check for the sign.Initially,initially with positive 1.
int sign = 1;

"variable to" is superfluous -- we already know they're variables :)

// try/catch block to check the bounds of Integer datatype.
try{
    int check = Integer.valueOf(str);
}
catch(NumberFormatException e){
    System.out.println("Number is going out of bounds of integer
           datatype.");
    System.exit(0);
}

What is the use in this?

Integer.valueOf converts a string to an integer. It does exactly what you're already trying to do!

So therefore I assume this exercise is some kind of study/homework? In which case relying on Integer.valueOf is a massive cop-out, and probably against the spirit of the code. I can see you're using it as a bounds check. But there are other, better ways to do that that will actually a) teach you something and b) not rely on a function that already does what your function is trying to do :)

If we go back to this comment you made:

  I have written this code to check all the edge cases in the implemention of an atoi() function.Please review.

Then we can see that you haven't checked anything! checkInt does it all for you! It's basically cheating :P So please remove that part of the code and try to think up a proper way to check the conditions checkInt is catching.

(Note: If this isn't a study piece, then the entire program should be replaced by a call to Integer.valueOf or even Integer.parseInt :))

for(int i=0; i<l; i++){

almost all style guides ask for a bit of space around binary operators:

for (int i = 0; i < l; i++){

It's hard to critic the loop, as I would like to say things such as "What happens if a character is a letter?", but it's hard to say that because of the checkInt cheat in the way, which protects you. So I'll comment more after you've fixed the code to remove that!

Here are some more tests for your function, however:

 assert 1234 == atoiFunc("1234")
 assert -1234 == atoiFunc("-1234")
 assert 0 == atoiFunc("0")

 atoiFunc("") # should raise an error
 atoiFunc(null) # should raise an error
 assert 0 == atoiFunc("+0")
 assert 0 == atoiFunc("-0")

 atoiFunc("1234abc") # should raise an error
 atoiFunc("abc1234") # should raise an error
 atoiFunc("12    34") # should raise an error
 assert 1234 == atoiFunc("+1234")
 atoiFunc("+12+34") # should raise an error
 atoiFunc("12-34") # should raise an error
 atoiFunc("1234+") # should raise an error
 atoiFunc("-1234+") # should raise an error
 atoiFunc("+1234-") # should raise an error
 atoiFunc("1234-") # should raise an error
 atoiFunc(" 1234 ") # should maybe raise an error. up to you.

 int x = atoiFunc("4294967296") # I'll let you figure this one out :)
 int y = atoiFunc("4294967297") # I'll let you figure this one out :)
 int z = atoiFunc("-4294967296") # I'll let you figure this one out :)
 int w = atoiFunc("-4294967297") # I'll let you figure this one out :)


Recursive function to get unique properties I just wanted to share my newly created JavaScript code. This code is responsible for passing back only the unique elements for a given array. This one is quite useful when you have a huge array and you store objects with huge arrays.

Use case: you want to sort out all unique object to do provide a filtering mechanism based on the chips!

Like the array is the following:

[
    {
        "name": "custom object1",
        "chips": [
            {
                "name": "hello"
            },
            {
                "name": "war"
            }
        ]
    },
    {
        "name": "custom object2",
        "chips": [
            {
                "name": "hello"
            },
            {
                "name": "option"
            }
        ]
    }
]

The function:

function getUnique(array, props, level = 0) {
    var retArray = [];
    array.forEach(function (element) {
        var index = props[level];
        if (Array.isArray(props) && props.length - 1 > level) {
            if (element[index] !== undefined) {
                var res = getUnique(element[index], props, level + 1);
                res.forEach(function (subelement) {
                    if (!retArray.some(function (arrayItem) {
                        return arrayItem === subelement;
                    })) {
                        retArray.push(subelement);
                    }
                });
            }
        } else {
            if (element[index] !== undefined && !retArray.some(function (arrayItem) {
                return arrayItem === element[index];
            })) {
                retArray.push(element[index]);
            }
        }
    });
    return retArray;
}

How to call the function:

getUnique(array, ['chips', 'name']);

That's it in a nutshell. What do you think? Any suggestions or advice?

The function is a bit repetitive, as both cases (iterating over sub array vs individual elements) check if the element is undefined and if the return array includes that value. That could potentially be abstracted to a separate function or simplified.

It appears that the following line:

var index = props[level];

Could be moved outside of the forEach function, since it doesn't depend on any of the variables defined inside the callback function.

If you know that the values to be compared will most likely be primitive values like strings or integers then Array.includes() could likely be used to replace the calls to retArray.some(). While it may not reduce the computational complexity, it would allow simplifying the code to check for existing values. Note that function isn't supported by IE and other older browsers, and is case-sensitive when comparing strings and characters1


Finding an element without pair in a list O(n**2) I'm practicing algorithms, and I just can not come up with a faster solution to this problem, but I'm not hitting the benchmark.
Problem: In a large list of large integers, I must find the one element, that is present in odd numbers. E.g. [1,1,1,1, 2,2, 3,3,3, 4,4,4,4].
I wrote several solutions, but I can not increase the speed of the execution.
import random

def testdata():
  space = []
  for i in range(10000):
    space = (
        space
        + [random.randint(0,1000000000)]
        * (random.randint(1,10) * 2)
    )
  odd = random.randint(0,1000000000)
  print(odd)
  space = space + [odd]
  random.shuffle(space)
  return space

def solution(A):
  A.sort()
  index = 0
  while True:
    count = A.count(A[index])
    if count%2:
      return(A[index])
    else:
      index = index + count

def solution_b(A):
  for elem in set(A):
    if A.count(A)%2:
      return(elem)

I'm not only looking for a better solution, but I'd appreciate it if someone explained, how to approach this kind of Big O notation problems. Without using pandas or numpy, etc.

This is not a review, but an extended comment.
The linear time/constant space solution is too well known to be spelled out again. However, here it goes.
XOR of two equal numbers is 0, and XOR of a number and 0 leaves the number unchanged. XOR is commutative and associative operation; we may perform it in any order we wish, and arrive to the same result. In other words, if we XOR all of them, each pair of numbers would cancel to 0, and the final result would be the number without a pair, the one we are looking for.


Computing the sum of all measures for a certain group type I have a method that receives 2 params:

first one is an array of objects of type { group1: string, group2: string, measure: number, ... }
second one is a string

I also have an enum GroupColumns
enum GroupColumns {
  ColumnTypeX = 'Column Type X',
  ColumnTypeY = 'Column Type Y',
  ColumnTypeZ = 'Column Type Z'
}

This method computes the sum of all the measures for a certain group type when the second parameter matches the second group and the first group is equal to any group found in GroupColumns enum.
    private ComputeSpecificColumnMeasure(data: Array<IDynamic<any>>, groupType: String): number {
        let measure: number = 0;
        data.forEach(el => {
            if (el.group2 === groupType && (el.group1 === GroupColumns.ColumnTypeX || el.group1 === GroupColumns.ColumnTypeY || el.group1 === GroupColumns.ColumnTypeZ )) {
                measure += el.measure;
            }
        });

        return measure;
    }

I do not like the verifications inside the if. If I somehow want to add more columns(unlikely) I would need to expand it, or create another method only to check if group1 is equal to any GroupColumns.
Any opinions?
How can I make this code scalable and optimized

If possible, I'd change the GroupColumns to be an array instead. That way, all you need to do is use .includes to see if an item is included in it. Other improvements can be:
Let TS automatically infer types when possible. You only need to denote the type of a variable or the type of a return value when TS can't do so on its own. TS can automatically determine that ComputeSpecificColumnMeasure returns a number. It can also determine that measure will be a number, so there's no need to denote the type of either of those.
Avoid String type The "String" type is something created via the String constructor, eg const str = new String('data');. This is almost never what you want - for a normal string, use the string (primitive) type instead. (TSLint rule: ban-types)
Avoid any You have data: Array<IDynamic<any>>. any should be avoided in TypeScript whenever possible, because it's so flexible - it loses type safety. If you don't know what the type will be, use unknown instead, which is type-safe.
For a small example of a problem with any:
const fn = (arg: any) => {
    console.log(arg.toFixed(2));
}

This will not throw a TS error, but it's likely to throw a runtime error unless the argument passed happens to be a number. Using unknown instead will throw a TS error - it'll force you to narrow the type first before calling a particular method on it.
const fn = (arg: unknown) => {
    if (typeof arg === 'number') {
        console.log(arg.toFixed(2));
    }
}

Even if the value that's any isn't being used in the function, it'd good practice to avoid any when possible anyway. (TSLint rule: no-unsafe-any)
Capitalization Conventionally, only a few things use PascalCase in JS:

Namespaces (eg React)
Classes
Enums, sometimes

A plain method should probably use the standard camelCase instead.
const groupColumns = [
    'Column Type X',
    'Column Type Y',
    'Column Type Z'
] as const; // Use "as const" to prevent automatic widening to `string[]`

private computeSpecificColumnMeasure(data: Array<IDynamic<unknown>>, groupType: string) {
    let measure = 0;
    data.forEach(el => {
        if (el.group2 === groupType && groupColumns.includes(el.group1)) {
            measure += el.measure;
        }
    });
    return measure;
}

Or, you could use .filter and reduce if you wished:
private computeSpecificColumnMeasure(data: Array<IDynamic<unknown>>, groupType: string) {
    return data
        .filter(el => el.group2 === groupType && groupColumns.includes(el.group1))
        .reduce((measureSoFar, el) => measureSoFar + el.measure, 0);
}


Saving search history with many checks and validations inside I am absolutely confused by saveSearchHistory function. It seems dirty and unobvious.
Should I refactor it, and how?

SearchHistory.java

public class SearchHistory {
  public SearchHistory(Long userId, Long groupId, String searchString) {
        this.userId = userId;
        this.groupId = groupId;
        this.searchString = searchString;
    }
  ...
  //getters + setters
  ...
  public void validate() throws InvalidParameterException{
    if (searchString == null || searchString.length() < MIN_WORD_SIZE) {
      throw new InvalidParameterException("Wrong search word min size: "+searchString);
    }

    if (searchString.length() > MAX_WORD_SIZE) {
        searchString = searchString.substring(0, MAX_WORD_SIZE - 1);
    }
}
}

SearchHistoryController.java

@Repository
public class SearchHistorControllerImpl implements SearchHistoryController {

    @PersistenceContext(unitName = "userPU")
    private EntityManager userEm;

    @Override
    @Transactional
    public List<SearchHistory> getSearchHistories(Long userId) {
        return emptyIfNull(userEm.createNamedQuery("getSearchHistories")
                .setParameter("userId", userId)
                .getResultList());
    }

    @Override
    @Transactional
    public SearchHistory saveSearchHistory(Long userId, Long groupId, String word) throws InvalidParameterException{
        SearchHistory searchHistory = new SearchHistory(userId, groupId, word);
        searchHistory.validate(); //throws InvalidParameterException

        removeSameWord(userId, groupId, word);
        checkWordsSizeLimit(userId, groupId);

        searchHistory = userEm.merge(searchHistory);
        userEm.flush();
        return searchHistory;
    }

    private void removeSameWord(Long userId, Long groupId, String word) {
        userEm.createNamedQuery("removeSameWord")
                .setParameter("userId", userId)
                .setParameter("groupId", groupId)
                .setParameter("searchString", word)
                .executeUpdate();
    }

    private void checkWordsSizeLimit(Long userId, Long groupId){
        List<SearchHistory> searchHistoryIds = userEm.createNamedQuery("findOldSearchWordsIds")
                .setParameter("userId", userId)
                .setParameter("groupId", groupId)
                .setFirstResult(MAX_HISTORY_SIZE - 1)
                .getResultList();

        if (!searchHistoryIds.isEmpty()) {
            userEm.createNamedQuery("removeOldWords")
                    .setParameter("searchHistoryIds", searchHistoryIds)
                    .executeUpdate();
        }
    }
}

SearchHistory.html.xml

...
<query name="findOldSearchWordsIds">
    SELECT id
     FROM SearchHistory as searchHistory
     WHERE
        searchHistory.userId = :userId
        AND searchHistory.groupId = :groupId
     ORDER BY searchHistory.searchTime DESC
</query>

<query name="removeOldWords">
    DELETE FROM SearchHistory as searchHistory
    WHERE searchHistory.id in ( :searchHistoryIds )
</query>

<query name="removeSameWord">
    DELETE FROM SearchHistory as searchHistory
    WHERE
        searchHistory.searchString = :searchString
        AND searchHistory.userId = :userId
        AND searchHistory.groupId = :groupId
</query>
...

FullSearch.java

 ...
public void commandExecute() {
    buildSearchResult();

    if (whetherSaveRequestInHistory) {
        final String filteredString = StringUtils.removeEnd(filterString, "*");
        try {
            searchHistoryManager.saveSearchHistory(context.getUserId, groupId, filteredString);
        } catch (InvalidParameterException ipe) {
            log.info("Unable to save Search History to db: "+ ipe.getMessage());
        }
    }
}

...

It seems dirty and unobvious.

I wouldn't go that far. I think that the code generally looks fine.

Style

The indentation and spacing is generally fine, but sometimes it is inconsistent (2 vs 4 spaces, spaces around +).

validate

I would validate right in the constructor instead of in a public method, because someone might forget to call the method, and then you end up with an invalid object.

And as validate has side-effects (it doesn't just check the string, it actually changes it), I would add a JavaDoc comment.

removeSameWord

Is this method really needed? Your database schema should be able to use a unique constraint to not allow duplicate rows. Or, since you are using merge, you should also be able to just remove it completely, since merge updates existing entries.

Naming

Sometimes, you use word or words and sometimes searchString for what seems to me to be the same thing. This is a bit confusing, you should use variable names consistently.
the query removeOldWords could just be removeByIds, because it doesn't care if words are old or new, it just deletes all words with the given ids.
checkWordsSizeLimit doesn't just check, it also deletes, the name should reflect this (maybe something like enforceWordsSizeLimit; or split the method up in getLastIds and removeByIds).
the query removeSameWord could be renamed to remove (the query doesn't really care if it's the same word as some other word, it just removes the given entry).
removeSameWord could also be remove (the parameters make it clear what will be removed) or removeExisting.


Proper class user.php I'm learning to program with PHP, so I ask your help to check if my class user.php is ok.

include('password.php');
class User extends Password{

    private $_db;

    function __construct($db){
        parent::__construct();

        $this->_db = $db;
    }

  private function get_user_hash($username){
    try {
      $stmt = $this->_db->prepare('SELECT password FROM members WHERE username = :username AND active="Yes" ');
      $stmt->execute(array('username' => $username));
      $row = $stmt->fetch();
      return $row['password'];
      } catch(PDOException $e) {
      echo '
'.$e->getMessage().'
';
    }
  }

  public function login($username, $password){
    $hashed = $this->get_user_hash($username);
    if($this->password_verify($password,$hashed) == 1){
       $_SESSION['loggedin'] = true;
      return true;
      }
    }

  public function logout(){
    session_destroy();
    }

  public function is_logged_in(){
    if(isset($_SESSION['loggedin']) && $_SESSION['loggedin'] == true) {
      return true;
      }
    }

  public function is_logged_admin(){
    if(isset($_SESSION['level']) && $_SESSION['level'] == 2 ) {
      return true;
      }
    return false;
    }

  public function get_user_info($username){
    try {
      $stmt = $this->_db->prepare('SELECT * FROM members WHERE username = ?');
      $stmt->execute(array($username));
      return $stmt->fetch();
    } catch(PDOException $e) {
      echo '
'.$e->getMessage().'
';
      }
  }

  public function members_count(){
    try {
      $stmt = $this->_db->prepare('SELECT * FROM members ORDER BY memberID');
      $stmt->execute();
      $count = $stmt->rowCount();
      print("$count");
      } catch(PDOException $e) {
      echo '
'.$e->getMessage().'
';
      }
    }

  public function delete_user($id){
    try {
      $stmt = $this->_db->prepare('DELETE FROM members WHERE memberID = :id');
      $stmt->bindParam(':id', $_GET['id'], PDO::PARAM_INT);
      $stmt->execute();
      } catch(PDOException $e) {
      echo '
'.$e->getMessage().'
';
      }
    }

  public function update_account($username){
    try {
      $stmt = $this->_db->prepare('UPDATE members SET location = ?, bio = ?, relationship = ?, interested = ?, work = ? WHERE username = ?');
      $stmt->execute([$_POST['location'], $_POST['bio'], $_POST['relationship'], $_POST['interested'], $_POST['work'], $username]);
    } catch(PDOException $e) {
      echo '
'.$e->getMessage().'
';
      }
  }

//close
}

In terms of the php code itself, the only thing I notice is your use of the $_POST superglobal in your class. This isn't wrong per say but I typically try to avoid use of $_POST in my objects, and instead pass the data in via parameters. Keeping all of your $_POST access in the same place makes it easier to manager changes, especially this the contents of $_POST is so easy to change. Same is also true for $_GET and $_REQUEST.

The same is also kind of true for the $_SESSION variable. It's not as important where it is, but stick to access within a limited set of locations.

Here's a few points to consider in terms of how you've constructed your class. They're not really php things, more good object construction practice.

Make sure you're always returning something from your methods. In many you have a return if things go right, but nothing if things go wrong. This will result in a NULL being returned but it's better practice to define this for reliable and consistent operation.

To take a more MVC approach, and to make your code easier to change and maintain, you shouldn't be echoing your errors in the class. Rather set the message to an error variable that can be retrieved by the calling code if needed, and then return FALSE. Echoing the error as it happens can be good during development but I recommend against it for production. If you still do want to see the error, best to handle it in the calling code so that you can easily change how the error is displayed. You could use a debugMode flag in the class if you want the best of both.

Your code indenting could really be improved.

private function get_user_hash($username){
    try {
        $stmt = $this->_db->prepare('SELECT password FROM members WHERE username = :username AND active="Yes" ');
        $stmt->execute(array('username' => $username));
        $row = $stmt->fetch();
        return $row['password'];
    } catch(PDOException $e) {
        echo '
'.$e->getMessage().'
';
    }
}

Lastly I recommend using CamelCase for your method names. The use of lowercase and underscores to me is a bit oldschool and more difficult to read.

private function getUserHash($username)


Evaluating a tic-tac-toe board using a C-style for loop in Rust I program in C, and it's my favourite language. I've recently found Rust. I like their ideas. Sometimes looks like a weird mixture of C++ and Haskell, but overall it looks nice.

The for loop in C is very flexible, and I truly miss them in Rust. Just look at my messy beginner Rust code below. Below that there is the equivalent C version, which (hopefully) doesn't look 'that' bad. I put the C version because I know my Rust code is unreadable; you need to look at C to understand why I wrote that mess. Tell me how to make my Rust code better. It doesn't have to copy all the C patterns, but there really needs to be something to compensate the lack of the flexible for loop.

If you're interested in what the whatever my code does. It determines the result of a tic-tac-toe board.

#![feature(libc)]

extern crate libc;

const SIZE: usize = 3;
const SIZE_2: usize = SIZE * SIZE;

fn determine_result(board: &[i32; SIZE_2]) -> i32 {
  'out: loop {
    let mut i = 0;
    'a: while i < SIZE_2 {
      if board[i] == 0 {
        i += SIZE;
        continue;
      }
      for j in i + 1 .. i + SIZE {
        if board[i] != board[j] {
          i += SIZE;
          continue 'a;
        }
      }
      return board[i];
    }
    'b: for i in 0 .. SIZE {
      if board[i] == 0 {
        continue;
      }
      let mut j = i + SIZE;
      while j < i + SIZE_2 {
        if board[i] != board[j] {
          continue 'b;
        }
        j += SIZE;
      }
      return board[i];
    }
    i = SIZE + 1;
    while i < SIZE_2 {
      if board[i] != board[0] {
        break 'out;
      }
      i += SIZE + 1;
    }
    return board[0];
  }
  let mut i = SIZE * 2 - 2;
  while i <= SIZE_2 - SIZE {
    if board[i] != board[SIZE - 1] {
      return 0;
    }
    i += SIZE - 1;
  }
  return board[SIZE - 1];
}

const N: usize = 50000000;

fn main() {
  unsafe {
    libc::srand(0);
  }
  let mut n = 0 as usize;
  let mut board: [i32; SIZE_2] = [0; SIZE_2];
  for i in 0 .. N {
    for i in 0 .. SIZE_2 {
      unsafe {
        board[i] = libc::rand() as i32 % 3;
      }
    }
    n += determine_result(&board) as usize;
  }
  println!("{}", n);
}

#include <stdio.h>
#include <stdlib.h>

#define SIZE 3
#define SIZE_2 (SIZE * SIZE)

static int determineResult(int board[static SIZE_2]) {
  for (int i = 0; i < SIZE_2; i += SIZE) {
    if (!board[i]) {
      continue;
    }
    for (int j = i + 1; j < i + SIZE; ++j) {
      if (board[i] != board[j]) {
        goto next;
      }
    }
    return board[i];
  next:;
  }
  for (int i = 0; i < SIZE; ++i) {
    if (!board[i]) {
      continue;
    }
    for (int j = i + SIZE; j < i + SIZE_2; j += SIZE) {
      if (board[i] != board[j]) {
        goto next2;
      }
    }
    return board[i];
  next2:;
  }
  for (int i = SIZE + 1; i < SIZE_2; i += SIZE + 1) {
    if (board[i] != *board) {
      goto next3;
    }
  }
  return *board;
next3:
  for (int i = SIZE * 2 - 2; i <= SIZE_2 - SIZE; i += SIZE - 1) {
    if (board[i] != board[SIZE - 1]) {
      return 0;
    }
  }
  return board[SIZE - 1];
}

#define N 100000000

int main(void) {
  srand(0);
  size_t n = 0;
  for (long i = 0; i < N; ++i) {
    int board[SIZE_2];
    for (int i = 0; i < SIZE_2; ++i) {
      board[i] = rand() % 3;
    }
    n += determineResult(board);
  }
  printf("%zu\n", n);
  return EXIT_SUCCESS;
}

I'm going to start with the tough love... both your C and Rust code are unreadable. Nested loops with goto statements is the definition of spaghetti code, and that tends to be the least understandable and unmaintainable type of code. The good news is that you are here, asking for feedback, which is the right direction! ^_^

I'm going to mark suggestions with [universal] if they are just general programming ideas, not specific to Rust.

Rust uses a 4 space indent, not 2.
Ranges don't have spaces. Use 0..20 instead of 0 .. 20.
Don't use the libc crate for random numbers, use the dedicated rand crate. This allows you to not need any unsafe code.
[universal] Always use a random number function for ranges. You should not use the modulus operator % as it can introduce sampling bias, preventing you from having a uniform distribution. In Rust, you can use gen_range.
Use _ when you want to ignore unused variables, like the index value of a loop:

for _ in 0..10 { }

[universal] Program-wide constants like N are terrible names. Give them better names that convey meaning and usage:

const BENCHMARK_ITERATIONS: usize = 50000000;

const SIZE_OF_ROW: usize = 3;
const SIZE_OF_BOARD: usize = SIZE_OF_ROW * SIZE_OF_ROW;

Don't specify the type of a variable when it can be inferred:

let mut board = [0; SIZE_2]; // no `: [i32; SIZE2] on the left

n += determine_result(&board); // no `as usize`

Use iterators instead of manually incrementing an array index. This has improved performance, and prevents you from duplicating the length variable everywhere:

for space in board.iter_mut() {
    *space = 0;

[universal] Declare variables with the minimal scope needed to do the job. For example, your board is only used inside the loop:

for i in 0..10 {
    let mut board = [0; SIZE_2];

Use iterator adapters, like map or fold, instead of doing all the work yourself. These have higher semantic meaning, allowing other programmers to instantly know you are transforming a sequence or reducing it down to a single value. It also allows your code to avoid some mutability.
[universal] Extract out functions, such as one to generate a board. This allows the mutability of the board to be constrained to a very small area.
I'd create a newtype for Board, or at the very least a type alias. This allows you to have higher concepts as opposed to just an array.
There's no need for explicit return statements at the end of a function.
[universal] Extract out more functions. determine_result has too much going on in it, and breaking them out into functions makes the code way easier to reason about. Functions for "row winners" or "column winners" are identifiable.
[universal] There's no reason to use an i32 for 3 values, that's massive overkill.
The absence of a value isn't really just another value, and you should use Option instead.

All together now:

src/main.rs

extern crate rand;
extern crate itertools;

use rand::{StdRng, Rng, SeedableRng};
use itertools::Itertools;

const SIZE_OF_ROW: usize = 3;
const SIZE_OF_BOARD: usize = SIZE_OF_ROW * SIZE_OF_ROW;

type Board = [Option<Piece>; SIZE_OF_BOARD];

#[derive(Debug, Copy, Clone, PartialEq)]
enum Piece {
    X,
    O,
}

// If every item in an iterator is the same, that value is
// returned. If there are no items or not all items are equal, returns
// None.
fn all_the_same<I>(mut iter: I) -> Option<I::Item>
    where I: Iterator,
          I::Item: PartialEq + ::std::fmt::Debug
{
    let initial_value = match iter.next() {
        Some(v) => v,
        None => return None,
    };

    while let Some(value) = iter.next() {
        if initial_value != value {
            return None
        }
    }

    Some(initial_value)
}

fn flatten<T>(opt: Option<Option<T>>) -> Option<T> {
    match opt {
        Some(inner) => inner,
        None => None,
    }
}

fn column_winner(board: &Board) -> Option<Piece> {
    (0..3).filter_map(|col| {
        flatten(all_the_same(board.iter().skip(col).step(3)).cloned())
    }).next()
}

fn row_winner(board: &Board) -> Option<Piece> {
    board.chunks(3).filter_map(|row| {
        flatten(all_the_same(row.iter()).cloned())
    }).next()
}

fn top_left_diagonal_winner(board: &Board) -> Option<Piece> {
    flatten(all_the_same(board.iter().step(4)).cloned())
}

fn top_right_diagonal_winner(board: &Board) -> Option<Piece> {
    flatten(all_the_same(board[2..7].iter().step(2)).cloned())
}

fn determine_result(board: &Board) -> Option<Piece> {
    column_winner(board)
        .or_else(|| row_winner(board))
        .or_else(|| top_left_diagonal_winner(board))
        .or_else(|| top_right_diagonal_winner(board))
}

const BENCHMARK_ITERATIONS: usize = 50000000;

fn generate_board<R>(mut rng: R) -> Board
    where R: Rng
{
    let mut board = [None; SIZE_OF_BOARD];
    for space in board.iter_mut() {
        *space = match rng.gen_range(0, 3) {
            1 => Some(Piece::X),
            2 => Some(Piece::O),
            _ => None,
        };
    }
    board
}

fn main() {
    let mut rng = StdRng::from_seed(&[0]);

    let total =
        (0..BENCHMARK_ITERATIONS)
        .map(|_| generate_board(&mut rng))
        .map(|board| determine_result(&board))
        .map(|winner| {
            match winner {
                Some(Piece::X) => 1,
                Some(Piece::O) => 2,
                None => 0,
            }
        })
        .fold(0, |acc, val| acc + val);

    println!("{}", total);
}

Cargo.toml

[package]
name = "tictactoe"
version = "0.1.0"
authors = ["A. Developer <a.developer@example.com>"]

[dependencies]
rand = "*"
itertools = "*"


IKM C++ assessment asks about std::unordered_set, part #1 Recently I had to do an IKM C++ assessment to qualify for a job. Some of the questions seemed strange to me, so I'll be doing a few posts here to discuss.
The first one is related to use of std::unordered_set. From what I remember it went something like:

A programmer is using a struct to contain some data. The struct must be stored in a std::unordered_set. True or false: The structure has to overload the method operator==() in order for the structure to be supported by std::unordered_set?

I wrote a test program to demonstrate:
#include <iostream>
#include <random>
#include <unordered_set>

struct ikm_test_struct {

   size_t m1;

   std::size_t operator()(ikm_test_struct const& s) const noexcept;

   constexpr bool operator==(const ikm_test_struct &rhs) const;
};

// Method operator() must return a size_t so use its limits in our distribution.

std::uniform_int_distribution<> distrib(
   std::numeric_limits<std::size_t>::min(),
   std::numeric_limits<std::size_t>::max()
);

std::size_t ikm_test_struct::operator()(ikm_test_struct const& s) const noexcept {

   // Seed a random number generator using the value in the parameter struct.

   std::mt19937 gen(s.m1);

   // Exercise the random number generator so that it is well away from any less random starting values.

   for (size_t i = 0; i < 100; ++i) distrib(gen);

   // Generate a random number in the required range.

   return distrib(gen);
}

constexpr bool ikm_test_struct::operator==(const ikm_test_struct &rhs) const {
   return m1 == rhs.m1;
}

typedef std::unordered_set<ikm_test_struct, ikm_test_struct> ikm_test_unordered_set;

int main()
{
   ikm_test_unordered_set x;

   for (size_t i = 0; i < 100; ++i) {

      ikm_test_struct y;
      y.m1 = i;

      x.insert(y);
   }

   size_t j = 0;
   for (const auto& k: x) {
      std::cout << "x[" << j << "] = " << k.m1 << std::endl;
      ++j;
   }
}

So it is possible to write a structure that implements operator==() and is then used in a std::unordered_set. Given this demonstration how should we answer the original question:

The demo code proves that it is possible to do this but the question says "has to overload the method operator==()". There are other ways to implement an equality test. Like creating a separate class or struct and using that class as the third parameter on the std::unordered_set template. So do you answer the question yes because it is possible or no because you don't have to do it that way?
The question says "has to overload". But the demo code implements operator==() only once, so there is no "overloading" of this method. If there is no overloading should the question be answered false? That seems to miss the point of asking the question to test your C++ knowledge.
The question says "in order for the structure to be supported". "Supported" is kind of ambiguous. Supported how, with what specialization of std::unordered_set, with what other helper classes?

Given the above it seems that the question is one that has no correct answer. Therefore, it is stupid to include it in a test that is supposed to evaluate someones C++ skills.
What do you think?

Defining an operator== for your own type is overloading.  The overload set includes all the built-in signatures.
Based on how you worded it, I agree with (1).  It is not true that you must define operator== for that struct in order to use it with std::unordered_set.  You can use the template argument, or you can cause the default predicate to work in other ways especially in newer versions of C++ (e.g. default the spaceship operator).
As for the example, doesn't it also need to specialize std::hash in order for it to work?  std::unordered_set<ikm_test_struct, ikm_test_struct> doesn't make sense.
If this test is going to be read by other prospective team members, then essaying what you thought is going to communicate to them that you (1) know your stuff; and (2) will be likely to apply the same skill to interpreting requirements.  Finish by correcting the sloppy nomenclature, linking to cppreference and/or a copy of the Standard, and reviewing the code of the sample (with citations to the Standard Guidelines).
If you're passed on as "overqualified", you'll be glad in the long run.
If it's a multiple-choice computer-based thing that just reports the score, that can be a real pain.  I recall encountering one where the expected results were sometimes just plain wrong, or out of date, or the question was worded poorly.  You should contact the people who made the test.


Extracting First, Last & Middle Name from a full name My function takes a string of Name and returns First Name, Last Name and Middle Name as output.

Please review and help me optimize it.

    //input --> Hamza Khan
    ProcessFullName(full_name): Object {
        var Obj = {
            FirstName: "",
            LastName: "",
            MiddleName: ""
        };
        var splitName = full_name.split(" ");
        if (splitName.length>0) {
            switch (splitName.length) {
                case 1:
                    Obj.FirstName = splitName[0];
                    break;
                case 2:
                    Obj.FirstName = splitName[0];
                    Obj.LastName = splitName[1];
                    break;
                case 3:
                    Obj.FirstName = splitName[0];
                    Obj.MiddleName = splitName[1];
                    Obj.LastName = splitName[2];
                    break;
            }
        }
        return Obj;
    } //Output --> { FirstName: "Hamza", LastName: "Khan", MiddleName: "" }

Provide a type for your full_name argument and a more specific type than Object for the return type:

interface SplitNameData {  // Does anybody have a better interface name?
  FirstName: string;
  MiddleName: string;
  LastName: string;
}

ProcessFullName(full_name: string): SplitNameData {
  const Obj: SplitNameData = {
    firstName: '',
    middleName: '',
    lastName: ''
  };

I think it would be cleaner to use null as a marker for non-existing names instead of the empty string even though the empty is probably not a valid name anywhere in the world.
What is your function supposed to do with names having more than three parts?
Consider using const wherever possible and let in all other cases. I recommend to never use var. ⇒ const Obj, const splitName.
Use a more descriptive name for Obj, e.g. splitNameData or separatedNames. In this case, I would recommend to rename splitName to something like nameParts as well to avoid confusion.
Use a consistent naming scheme: fullName instead of full_name, firstName instead of FirstName and so on.
Use array destructuring:

case 2:
  [Obj.FirstName, Obj.LastName] = splitName;
  break;
case 3:
  [Obj.FirstName, Obj.MiddleName, Obj.LastName] = splitName;
  break;

ad performance: for this computationally simple task, any optimization would be premature optimization in my opinion.
Alternative (maybe cleaner?) solution using named RegExp capture groups:

const NAME_PATTERN = /^(?<firstName>[^ ]*) ((?<middleName>[^ ]*) )?(?<lastName>[^ ]*)$/;

const emptyNameData = {
  firstName: '', middleName: '', lastName: ''
};

const {
  groups: {
    firstName,
    middleName = '',
    lastName
  } = emptyNameData
} = NAME_PATTERN.exec(fullName) || {};
alert([firstName, middleName, lastName]);

I am not very keen on the || {} part, maybe an if-else checking the match object for null would be a better solution.


Javascript to compute max width and height for a nested HTML element I wrote the following code to give an HTML element max width/height in its parent container.
I think I address the box model issues but I am wondering if anyone can see issues and or shortcomings.

Are there other solutions out there that accomplish this? I want to make sure I didn't re-invent the wheel.

function heightBuffer(control) {
    return parseInt((control.outerHeight(true)) - control.height());
}

function widthBuffer(control) {
    return parseInt((control.outerWidth(true)) - parseInt(control.width()));
}

function MaxHeightInParent(control, minHeight, controlHeightsToSubtract) {
    var h = parseInt(control.parent().height()) - heightBuffer(control);
    if (controlHeightsToSubtract != null)
        $.each(controlHeightsToSubtract, function(index, value) {
            h = h - parseInt(value.outerHeight(true));
        });

    if (minHeight != null && h < minHeight) h = minHeight;
    control.height(0);
    control.css('min-height', h);
}

function MaxWidthInParent(control, minWidth, controlWidthsToSubtract) {
    var w = parseInt(control.parent().width()) - widthBuffer(control);
    if (controlWidthsToSubtract != null)
        $.each(controlWidthsToSubtract, function(index, value) {
            w = w - parseInt(value.outerWidth(true));
        });
    if (minWidth != null && w < minWidth) w = minWidth;
    control.width(0);
    control.css('min-width', w);
}

Note controlHeightsToSubtract / controlWidthsToSubtract are if you wish to pass an array of controls that share the containing element with the element you are attempting to maximize in Height/Width.

What is

parseInt((control.outerHeight(true)) - control.height());

supposed to achieve?

The result of - is always going to be a number, so what this does is convert the number to a string and back to a number.  This can only lose precision and waste time.

Similarly, the use of parseInt on the first argument is also unnecessary

parseInt(control.parent().height()) - heightBuffer(control)

since

"4" - 1 === 3

In fact, the automatic conversion that - does is better than that done by parseInt since parseInt falls back to octal on some interpreters but not others.

"10"             - 1 === 9
parseInt("10")   - 1 === 9
"0x10"           - 1 === 15
parseInt("0x10") - 1 === 15
"010"            - 1        // throws reliably
parseInt("010")  - 1 === 7  // on some and 9 on others

so I'd get rid of all the uses of parseInt as an operand to -.

You seem to be setting min-height but the method is called MaxHeight.  That confuses me.

When you change the CSS

control.css('min-height', h)

you might want to specify units as in

control.css('min-height', h + "px")

I'm not sure how you're handling the widths of margins and borders on the controls.  Is that important to you?


40 million tweets from 200k users mysql> select count(username) from users where no_tweets  > 50;
+-----------------+
| count(username) |
+-----------------+
|          282366 |
+-----------------+
1 row in set (5.41 sec)

mysql> select sum(no_tweets) from users where no_tweets > 50;
+----------------+
| sum(no_tweets) |
+----------------+
|       38569853 |
+----------------+
1 row in set (1.75 sec)

I have that many users, who have collectively tweeted that many tweets. My aim is to store them in a file and then find out what a user generally tweets about (for starters, my aim is to run vanilla LDA and see if it works well on short documents), but the problem is that I ran the python code like an hour back and it has still not finished even 2% of users. I have posted the python code below.

'''
The plan is : Find out all users  from the users database who have more than 50 tweets
Create a file for them in the directory passed as an argument with the same name as the username and write all the tweets in them
'''
def fifty(cursor):
    ''' Find all users and return , all of them having more than 50 tweets'''
    cursor.execute("select username from users where no_tweets>50")
    return cursor.fetchall()

def connect():
    ''' Cursor for mySQLdb'''
    conn = MySQLdb.connect(host="localhost",user="rohit",passwd="passwd",db="Twitter")
    return conn.cursor()

def tweets(cursor,name):
    ''' All tweets by a given user'''
    cursor.execute("select tweets from tweets where username='"+name+"'")
    return cursor.fetchall()
import sys,os,MySQLdb
directory = sys.argv[1] #Directory to write the user files
cursor = connect()
rows = fifty(cursor) #Find all users who have more than 50 tweets

for i in rows:#For all users
    data = open(os.path.join(directory,i[0]),'w') #Open a file same as their name
    allTweets = tweets(cursor,i[0]) #Find all tweets by them
    for j in allTweets:
        data.write(j[0]+"\n") #Write them
    data.close()

The problem is that the code is running too slow and at this rate, it will take more than a day for it to finish writing all files. So some way that would make it faster would be great. So yeah, that is the question.

Yikes, you're running almost 3,000,000 individual queries.  If you could do 4 a second (and you probably cannot) that is still a day!

How about just

select username, tweets from tweets order by username

Then of course you have to do some fancy footwork to switch from one file to the next when the user changes.

It should only take a few hours to run.


Calling a Haskell MD5 hashing function from C, and returning the result back to C land I'm starting to learn how to meld the worlds of C and Haskell. Looking for any feedback on this first function.

The function takes in a pointer to an array of unsigned chars and returns a pointer to 32 unsigned shorts. Or at least I would like it too :).

I am unsure about when the memory used to return the result is cleaned up. Is it possible that it will be garbage collected before I can use it on the C side?

Anyway here's the code.

{-# LANGUAGE ForeignFunctionInterface #-}
module JSFLPlugin where
import Foreign.C.Types (CInt(..))
import Foreign.ForeignPtr (newForeignPtr_)
import Foreign.Ptr (Ptr)
import Data.Serialize (encode)
import Foreign.ForeignPtr.Unsafe (unsafeForeignPtrToPtr)
import Data.Digest.Pure.MD5 (md5)
import Data.ByteString (unpack)
import Data.ByteString.Internal (toForeignPtr, fromForeignPtr)
import Data.ByteString.Lazy.Internal (chunk)
import Data.ByteString.Lazy (empty)
import Data.Word (Word8, Word16)
import Data.Text.Format (hex)
import Control.Applicative ((<$>))
import Data.Text.Lazy.Builder (toLazyText)
import Data.Text.Lazy (toStrict)
import Data.Text.Foreign (asForeignPtr)
import Data.Monoid ((<>), mempty)

-- | Hash pointer to array of unsigned chars
hash :: CInt -> Ptr Word8 -> IO (Ptr Word16)
hash count addr =  do
    -- cast to ForeignPtr
    fptr <- newForeignPtr_ addr
    -- Make a strict ByteString
    let sbyte    = fromForeignPtr fptr 0 (fromIntegral count)
        -- Make a lazy ByteString from the strict one
        lbyte    = chunk sbyte empty
        -- Hash with md5 and encode as a strict ByteString
        digest   = encode . md5 $ lbyte
        -- Convert the each digest byte to a UTF-16 encoded hexdecimal character
        hexBytes = toLazyText . foldr (\x y -> y <> hex x) mempty . unpack $ digest
    -- Convert to a strict Text and get the pointer to chars
    fmap (unsafeForeignPtrToPtr . fst) . asForeignPtr . toStrict $ hexBytes

foreign export ccall hash :: CInt -> Ptr Word8 -> IO (Ptr Word16)

EDIT: New version per Joey's suggestions (compiled but not tested).

{-# LANGUAGE ForeignFunctionInterface #-}
module JSFLPlugin where
import Foreign.C.Types (CInt(..))
import Foreign.ForeignPtr (newForeignPtr_)
import Foreign.Ptr (Ptr)
import qualified Data.Serialize as Serialize
import Data.Digest.Pure.MD5 (MD5Digest)
import Data.ByteString (ByteString)
import Data.ByteString.Internal (fromForeignPtr)
import Data.Word (Word8, Word16)
import Data.Text (Text)
import Data.Text.Foreign (unsafeCopyToPtr)
import qualified Data.ByteString.Base16 as Base16
import Data.Text.Encoding (decodeUtf8)
import Crypto.Classes (hash')

pureHash :: ByteString -> Text
pureHash = decodeUtf8 . Base16.encode . Serialize.encode . md5 where
    md5 s = hash' s :: MD5Digest

hash :: CInt -> Ptr Word8 -> Ptr Word16 -> IO ()
hash count input output =  do
    fptr <- newForeignPtr_ input
    let sbyte = fromForeignPtr fptr 0 $ fromIntegral count
    unsafeCopyToPtr (pureHash sbyte) output

foreign export ccall hash :: CInt -> Ptr Word8 -> Ptr Word16 -> IO ()

First, you have way too many comments.  They clutter the code, and some of them are completely redundant.

-- | Hash pointer to array of unsigned chars
hash :: CInt -> Ptr Word8 -> IO (Ptr Word16)

The comment provides no information beyond the function's name and signature.  But it doesn't answer these questions:

What hashing algorithm is being used?
Why is the output Ptr Word16?  UTF-16?
Is the output in hexadecimal, or raw bytes?  If it's hexadecimal, does it use capital or lowercase letters?

Before we dive into foreign API details, we can do a little refactoring.  There's a big chain of pure function application here.  Let's move it outside so it doesn't clutter the scary foreign pointer stuff:

hashPure :: ByteString -> Text
hashPure sbyte =
    let lbyte    = chunk sbyte empty
        digest   = encode . md5 $ lbyte
        hexBytes = toLazyText . foldr (\x y -> y <> hex x) mempty . unpack $ digest
     in toStrict hexBytes

I think we can improve this a bit.  Let's first take a step back.  Here's all hashPure needs to do:

Hash the input string (a strict ByteString) using MD5
Convert it to a hex string (as Text)

To get there, we have to do a bunch of annoying conversions.  But we can avoid these if we tweak a couple of our library choices.

Because of the Hash instance on MD5Digest, we can eliminate a strict to lazy ByteString conversion:

import Crypto.Classes (hash')
import Data.Digest.Pure.MD5 (MD5Digest)

hashPure :: ByteString -> Text
hashPure sbyte =
    let digest   = encode (hash' sbyte :: MD5Digest)
        hexBytes = toLazyText . foldr (\x y -> y <> hex x) mempty . unpack $ digest
     in toStrict hexBytes

Your code for converting to hex is bogus.  To illustrate why:

>foldr (\x y -> y <> hex x) mempty [1,2,3]
"321"

Namely:

Your use of foldr flips the order.  What you meant was: foldr (\x y -> hex x <> y)
The hex function does not pad the number to two characters.

To avoid the first problem, you could have just said mconcat (map hex).  foldr/build fusion should optimize it.

To fix the second problem, and avoid two more conversions, you can use the base16-bytestring package.  You could avoid yet another conversion if you make hash return an array of chars instead of wide chars, but I won't do that yet.

hashPure :: ByteString -> Text
hashPure sbyte =
    let digest   = encode (hash' sbyte :: MD5Digest)
        hexBytes = Base16.encode digest
     in decodeUtf8 hexBytes

We can make it a little prettier using point-free style:

hashPure :: ByteString -> Text
hashPure =
    decodeUtf8 . Base16.encode . Serialize.encode . md5
  where
    md5 s = hash' s :: MD5Digest

Now on to the ForeignPtr stuff.  I saved the best for last.

hash :: CInt -> Ptr Word8 -> IO (Ptr Word16)
hash count addr =  do
    fptr <- newForeignPtr_ addr
    let sbyte = fromForeignPtr fptr 0 (fromIntegral count)
    unsafeForeignPtrToPtr . fst <$> asForeignPtr (hashPure sbyte)

Your two points of interaction with C:

Convert a C array of bytes to a ByteString
Convert a Text to a block of wide chars, and return a pointer to it.

Your first conversion is sound, but I would use unsafePackCStringLen instead (which does the same thing):

hash :: CInt -> Ptr Word8 -> IO (Ptr Word16)
hash count addr =  do
    sbyte <- unsafePackCStringLen (castPtr addr, fromIntegral count)
    unsafeForeignPtrToPtr . fst <$> asForeignPtr (hashPure sbyte)

The second conversion is wrong.  ForeignPtr is for managing C objects in Haskell, not the other way around.  The returned pointer will be invalidated when the ForeignPtr returned by asForeignPtr is garbage collected.

You fix this, you would need to malloc a large enough buffer, then copy the text content into it.  Unfortunately, Data.Text.Foreign does not provide a convenient way to do this without a needless allocation.

I would change the signature of the C function from this:

uint16_t *hash(int blksize, uint8_t *block);

To this:

void hash(uint8_t *block, size_t blksize, char out[33]);

Namely:

Write a string to a caller-supplied buffer of the correct size, rather than allocating a buffer that the caller has to free.  A hex-encoded MD5 hash is always 32 characters long.  The 33rd byte is for the zero terminator.
Output a C string instead of UTF-16.  Unless you plan to throw the output into a system call or something that takes UTF-16, you shouldn't be using UTF-16.
Flip pointer and size arguments.  This is mainly a matter of taste.

See if you can implement this part yourself.  This post is already too long.