Document distance in Python I wrote a script to calculate document distance. It seems working but I couldn't be sure. (I tried for the small strings and it seems working) Also I am not sure that its fast enough for large texts.

Here is the Document Distance formula:

The frequency is the number of occurrences for each object. Such as for
"D1: be or not to be" the frequency will be freq$$(D_1) = {be=2, not=1, or=1, to=2}$$ and $$||D_1|| = \sqrt{2^2 + 1^2 + 1^2 + 2^2}$$

from math import sqrt
from string import ascii_lowercase

alphanumerics = ascii_lowercase + "0123456789"

file1 = open("document1.txt", "r")
file2 = open("document2.txt", "r")
print(file1)

document1 = ""
for line in file1:
    document1 += line + ""

document2 = ""
for line in file2:
    document2 += line + ""

if document1 == "":
    print("---  The document1.txt file is empty! ---")
    raise ValueError
elif document2 == "":
    print("---  The document2.txt file is empty!! ---")
    raise ValueError

print(document1, document2)

def word_processor(document):
    '''returns an array that only contains the words in the text'''
    doc_word = []
    word = ""
    count_space = 0
    for char in document.lower():
        if char in alphanumerics:
            word += char
            count_space = 0
        elif char == " " and count_space == 0:
            doc_word.append(word)
            word = ""
            count_space += 1
    doc_word.append(word)
    return doc_word

doc1_words = word_processor(document1)
doc2_words = word_processor(document2)

dot_product = 0
doc1_word_freq = {i: doc1_words.count(i)
                for i in doc1_words}  # the doc1 vector
doc2_word_freq = {i: doc2_words.count(i)
                for i in doc2_words}  # the doc2 vector
for key, value in doc1_word_freq.items():
    if key in doc2_word_freq.keys():
        dot_product += doc2_word_freq[key] * value # the dot product of the two doc vectors

doc1_mag = sqrt(sum([value**2 for value in doc1_word_freq.values()])) # the magnitude of the each document
doc2_mag = sqrt(sum([value**2 for value in doc2_word_freq.values()]))

similarity = dot_product / (doc1_mag * doc2_mag) * 100
print("The similarity between 2 document is", similarity, "percent")

Any ideas ?

I did not use the acos function since I don't think I need it.

Most of your code can be greatly reduced.

Doing string addition is often a bad idea in Python, especially if you are doing it in a loop. Also note that a + "" == a for any string a. I would replace your mix of global code and the word_processor function with this short function:

import string

WHITELIST = set(string.ascii_letters + string.digits + string.whitespace)

def get_words(file_name):
    with open(file_name) as f:
        for line in f:
            line = "".join(c for c in line.casefold() if c in WHITELIST)
            yield from line.split()

This filters each line of the file to only contain ASCII letters and whitespace and splits it by whitespace afterwards (the default behavior of str.split). The yield from effectively flattens the output into a stream of words (the whole function is a generator, so you can consume the output one at a time without keeping the whole list in memory). It also uses the with keyword to ensure your files are closed, even in the event of an exception interrupting the program. I used str.casefold instead of str.lower, as recommended by @AJNeufeld in the comments.

You should put the actual code into a function as well. I would call it similarity, since it does not actually measure distance, but similarity (from 0 to 100). For this you can use collections.Counter instead of using list.count repeatedly (each of which is \$\mathcal{O}(n)\$).

from collections import Counter
from math import sqrt

def similarity(words1, words2):
    words1, words2 = Counter(words1), Counter(words2)
    dot = sum(words1[k] * words2[k] for k in words1.keys() & words2.keys())
    norm1 = sqrt(sum(v**2 for v in words1.values()))
    norm2 = sqrt(sum(v**2 for v in words2.values()))
    return dot / (norm1 * norm2)

Here I used the fact that Counter can take any iterable (like the output of get_words) and counts how often each item occurs (it returns a kind of dictionary). Also, dictionary keys behave like sets, so we can do a set intersection so we don't have to check for every key if it is also in the second words counter. I would also leave scaling the similarity to the caller.

Use it within a if __name__ == "__main__": guard to allow importing from the module. You might want to add a simple CLI interface with sys.argv (or do it properly and use argparse if you want more functionality). Throw in a f-string to make formatting the output to the user easier (and make it a percent with not too many digits shown).

import sys

if __name__ == "__main__":
    file1, file2 = sys.argv[1:3]
    sim = similarity(get_words(file1), get_words(file2))
    print(f"Similarity of {file1} and {file2}: {sim:.2%}")

Usage:

python3 similarity.py file.txt file_old.txt
# Similarity of file.txt and file_old.txt: 92.19%

For these two small files (21KiB), your code takes 0.3s, whereas my code only takes 0.05s on my machine.


Longest Increasing Subsequence I am learning dynamic programming and I have written down some code for longest increasing subsequence. I would like to know if there is any case or any area of improvement in the terms of optimization/programming style.

/**
 * LIS = Longest increasing subsequence.
 * Input = [10, 22, 9, 33, 21, 50, 41, 60, 80]
 * Output = [10, 22, 33, 50, 60, 80]
 * Created by gaurav on 1/7/15.
 */

function findSubsequence(arr){
    var allSubsequence = [],
        longestSubsequence = null,
        longestSubsequenceLength = -1;

    for(var i=0;i<arr.length;i++){          //i=1
        var subsequenceForCurrent = [arr[i]],
            current = arr[i],
            lastElementAdded = -1;
        for(var j=i;j<arr.length;j++){
            var subsequent = arr[j];
            if((subsequent > current) && (lastElementAdded<subsequent)){
                subsequenceForCurrent.push(subsequent);
                lastElementAdded = subsequent;
            }
        }
        allSubsequence.push(subsequenceForCurrent);
    }
    for(var i in allSubsequence){
        var subs = allSubsequence[i];
        if(subs.length>longestSubsequenceLength){
            longestSubsequenceLength = subs.length;
            longestSubsequence = subs;
        }
    }
    return longestSubsequence;
}

(function driver(){
    var sample = [87,88,91, 10, 22, 9,92, 94, 33, 21, 50, 41, 60, 80];
    console.log(findSubsequence(sample));
})();

This bit worries me:

lastElementAdded = -1;

It assumes that the minimum value in the array is zero. But really, an increasing sequence could start with -3456780 or something.

I'd use null or something non-numeric instead. Or simply add the first element automatically, and start the loop at index 1. You could use Number.NEGATIVE_INFINITY, but it has the same problem: A sequence could start at negative infinity.

Also, don't use for..in loops on arrays. A for..in loop enumerates an object's properties; it's not intended for iterating through an array's elements. Instead, use a regular for loop, or a forEach iterator.

Lastly, I'd change allSubsequence to allSubsequences (plural) simply because it more grammatically correct.

In terms of overall strategy, your current algorithm is doing a bit of unnecessary work. Given the example input, the first 3 subsequences it finds are:

[ 87, 88, 91, 92, 94 ]
[ 88, 91, 92, 94 ]
[ 91, 92, 94 ]

The last two aren't really interesting, since they're just subsequences of the first.

Now, this really isn't a problem for arrays as short as what you've got here. Still, it's a fun exercise, so I tried my hand at it. There's probably an even more elegant solution than what I'm proposing here, though. Algorithms aren't my strong suit, I'm afraid. But here's what I came up with:

Start a sequence with the first element of the input array
Iterate through the array

If a value is greater than the sequence's maximum, append it to sequence
If it's less and it's the first such value we've found, recurse with a subset of the input array, starting at the current index. Store the result.

Return whichever sequence - the current one, or the "fork" - is longer

Kinda hard to explain, actually. Hope the code below will help illustrate:

function findLongestIncreasingSequence(array) {
  var sequence = [],
      fork = null;

  // Always add the first value to the sequence
  sequence.push(array[0]);

  // Reduce the array with. Since no initial accumulator is given,
  // the first value in the array is used
  array.reduce(function (previous, current, index) {

    // If the current value is larger than the last, add it to the
    // sequence and return (i.e. check the next value)
    if(current > previous) {
      sequence.push(current);
      return current;
    }

    // If, however, the value is smaller, and we haven't had a fork
    // before, make one now, starting at the current value's index
    if(!fork && current < previous) {
      fork = findLongestIncreasingSequence(array.slice(index));
    }

    // Return the previous value if the current one is less or equal
    return previous;
  });

  // Compare the current sequence's length to the fork's (if any) and
  // return whichever one is larger
  return fork && fork.length > sequence.length ? fork : sequence;
}

Given the example input, you get:

findLongestIncreasingSequence(sample); // => [ 87, 88, 91, 92, 94 ]

Anyway, what it does is more or less this, where √ means the value was added to a sequence, and F means it "forked" and recursed

Initial input: 87  88  91  10  22  9  92  94  33  21  50  41  60  80
====================================================================
1st call:       √   √   √   F   .  .   √   √   .   .   .   .   .   .
2nd call:                   √   √  F   √   √   .   .   .   .   .   .
3rd call:                          √   √   √   F   .   .   .   .   .
4th call:                                      √   F   √   .   √   √
5th call:                                          √   √   F   √   √
6th call:                                                  √   √   √

Unfortunately, it's not tail-recursive, so call stack depth could be an issue.

And there's probably more optimizations that can be made. For instance, if the current sequence is already 5 items long, there's no reason to fork if the array only has 4 items left. And as I said, there's probably an even cleverer solution overall.

The alternative to the above would be to simply map out the indices at which a value is lower than the one preceding it. Then do the same thing (slice the array at those indices, find increasing sequence) one at a time instead of recursively, and compare sequence lengths at the end. Same result. But recursion is more fun :P


Best testing using a mocked repository [Test]
public void Post_ReturnsCorrectModel()
{
    var post = new Post
    {
        Slug = "continuing-to-an-outer-loop",
        Title = "Continuing to an outer loop",
        Summary = "When you have a nested loop, sometimes",
        Content = "When you have a nested loop, sometimes",
        PublishedAt = DateTime.Now.AddDays(7),
        Tags = new Collection<Tag> { new Tag { Name = "Programming" } }
    };

    repository.Setup(expression => expression.Find("continuing-to-an-outer-loop"))
              .Returns(post);

    var viewResult = (ViewResult) controller.Post("continuing-to-an-outer-loop");
    var actual = viewResult.Model as PostViewModel;

    Assert.NotNull(actual);

    Assert.AreEqual(actual.Slug, post.Slug);
    Assert.AreEqual(actual.Title, post.Title);
    Assert.AreEqual(actual.Summary, post.Summary);
    Assert.AreEqual(actual.Content, post.Content);
    Assert.AreEqual(actual.PublishedAt, post.PublishedAt);
    Assert.AreEqual(actual.Tags, post.Tags);
}

Hopefully the purpose of this test is already clear but to further clarify - the repository exposes a Post object that is mapped to a PostViewModel. I then proceed to assert that the action method returns the correct model.

The action method looks like this:

public ActionResult Post(string slug)
{
    Post post =
        repository.Find(slug);

    if (post == null || post.Draft)
    {
        return HttpNotFound();
    }

    return View("Post", post.MapTo<PostViewModel>());
}

Could you please review this test? My particular areas of concern are as follows:

Is the test well named?  (I adopted the name from  here).
Must I assert that each property is equal as I have above? I feel as though I can make a reasonable assumption that if one property is equal then the rest will be too.
Here I am modelling the post object based on a real blog post. I do not think that the value of each property really matters and that I could very well have used arbitrary strings. Are there any best practices regarding which test data to use?
Would it be reasonable to extract the post object to another class and expose it using a factory of sorts? This will reduce the ceremony of my test and keep things DRY. Are there any best practices regarding this?

I think the test is reasonably well named (imho).
If you assume you write the test against a black box (meaning you don't know how the controller code is implemented) then yes you should compare all properties. Someone could have forgotten to copy a property for example. This is especially the case for later when new properties get added (as a regression test). If you find yourself writing code like that a lot then there are several ways to automate this comparison.
I prefer to use human readable strings in unit tests. Makes it easier to read. However once you start doing integrations tests like hitting a test database you want to possibly generate some more exotic strings (non latin characters) to make sure that what you put into the db is also what you get back out and not some garbled nonsense. Some databases can be nasty in that regard.
If you find yourself creating the same post over and over again then you have some options:

If it's just inside one test fixture then create the post in the Setup routine.
If it's across multiple test fixtures then yes extract it into a test utility class.


Drawing random smooth lines contained within a square I'm trying to write a matlab function that creates random, smooth trajectories in a square of finite side length. Here is my current attempt at such a procedure:

function [] = drawroutes( SideLength, v, t)
%DRAWROUTES Summary of this function goes here
%   Detailed explanation goes here

%Some parameters intended to help help keep the particles in the box
RandAccel=.01;
ConservAccel=0;
speedlimit=.1;
G=10^(-8);
%
%Initialize Matrices
Ax=zeros(v,10*t);
Ay=Ax;
vx=Ax;
vy=Ax;
x=Ax;
y=Ax;
sx=zeros(v,1);
sy=zeros(v,1);
%
%Define initial position in square
x(:,1)=SideLength*.15*ones(v,1)+(SideLength*.7)*rand(v,1);
y(:,1)=SideLength*.15*ones(v,1)+(SideLength*.7)*rand(v,1);
%
for i=2:10*t
    %Measure minimum particle distance component wise from boundary
    %for each vehicle
    BorderGravX=[abs(SideLength*ones(v,1)-x(:,i-1)),abs(x(:,i-1))]';
    BorderGravY=[abs(SideLength*ones(v,1)-y(:,i-1)),abs(y(:,i-1))]';
    rx=min(BorderGravX)';
    ry=min(BorderGravY)';
    %
    %Set the sign of the repulsive force
    for k=1:v
    if x(k,i)<.5*SideLength
        sx(k)=1;
    else
        sx(k)=-1;
    end
    if y(k,i)<.5*SideLength
        sy(k)=1;
    else
        sy(k)=-1;
    end
end
%
%Calculate Acceleration w/ random "nudge" and repulive force
Ax(:,i)=ConservAccel*Ax(:,i-1)+RandAccel*(rand(v,1)-.5*ones(v,1))+sx*G./rx.^2;
Ay(:,i)=ConservAccel*Ay(:,i-1)+RandAccel*(rand(v,1)-.5*ones(v,1))+sy*G./ry.^2;
%
%Ad hoc method of trying to slow down particles from jumping outside of
%feasible region
for h=1:v
    if abs(vx(h,i-1)+Ax(h,i))<speedlimit
       vx(h,i)=vx(h,i-1)+Ax(h,i);
    elseif (vx(h,i-1)+Ax(h,i))<-speedlimit
       vx(h,i)=-speedlimit;
    else
        vx(h,i)=speedlimit;
    end
end
for h=1:v
    if abs(vy(h,i-1)+Ay(h,i))<speedlimit
       vy(h,i)=vy(h,i-1)+Ay(h,i);
    elseif (vy(h,i-1)+Ay(h,i))<-speedlimit
       vy(h,i)=-speedlimit;
    else
        vy(h,i)=speedlimit;
    end
end
%
%Update position
x(:,i)=x(:,i-1)+(vx(:,i-1)+vx(:,i))/2;
y(:,i)=y(:,i-1)+(vy(:,i-1)+vy(:,1))/2;
%
end
%Plot position
clf;
hold on;
axis([-100,SideLength+100,-100,SideLength+100]);
cc=hsv(v);
for j=1:v
plot(x(j,1),y(j,1),'ko')
plot(x(j,:),y(j,:),'color',cc(j,:))
end
hold off;
%
end

My original plan was to place particles within a square, and move them around by allowing their acceleration in the x and y direction to be governed by a uniformly distributed random variable. To keep the particles within the square, I tried to create a repulsive force that would push the particles away from the boundaries of the square.
In practice, the particles tend to leave the desired "feasible" region after a relatively small number of time steps (say, 1000)."

I'd love to hear your suggestions on either modifying my existing code or considering the problem from another perspective.

When reading the code, please don't feel the need to get hung up on any of the ad hoc parameters at the very beginning of the script. They seem to help, but I don't believe any beside the "G" constant should truly be necessary to make this system work.

Here is an example of the current output:

Many of the vehicles have found their way outside of the desired square region, [0,400] X [0,400].

Random walks "wander all around" by definition. So you have to twist the odds of the walk staying in the area you want.

It would be much simpler if you'd just make an attractor point at the center, that would move each point infinitesimally (some floating point value between 0 to 1 or sqrt(2) pixels) towards itself at each step.

Another option would be to make the edges exponentially hard to reach. I'm thinking of an Exponential_map with a really steep slope at the edges and otherwise near linear curve. also see Gamma curve. So essentially map an "infinite" area to the restricted space.

Third option would be to think about how magnetism works. I mean, a magnet can't be any more magnetized to a certain direction after all it's particles are magnetized towards that direction. And the magnetization takes exponentially stronger field all the way towards that state. I've written a "tape compression" algorithm with Numpy using that idea.


How secure is this PDO Portfolio Script? I was wondering if someone could take some time to look over my portfolio script and see let me know if it's secure enough to be uploaded live and connected to my database. My previous mySQL script was VERY insecure...

The Database Connection:

    public function  __construct($server, $databaseName, $username, $password)
    {
        $this->server = $server;
        $this->databaseName = $databaseName;
        $this->username = $username;
        $this->password = $password;
        $this->openConnection();
    }

    protected function openConnection()
    {
        $this->databaseLink = new PDO("mysql:host=$this->server;dbname=$this->databaseName", $this->username, $this->password);
    }

    public function getLink()
    {
        return $this->databaseLink;
    }
}

$database = new Database("localhost", "XXXXXXX", "XXXXXXX", "XXXXXXX");
$databaseLink = $database->getLink();
?>

The Client Detail Page

$client = $_GET["client"];

$getClientDataSql = "SELECT * FROM `client` WHERE `client`=:client";
$statement = $databaseLink->prepare($getClientDataSql);
$statement->bindParam(":client", $client, PDO::PARAM_STR);
$statement->execute();

if(!$statement->rowCount())
{
    echo "No Clients Found";
}
else
{
    $row = $statement->fetch(PDO::FETCH_ASSOC);

    // Title
    echo "
";
    echo "
".$row["title"]."
";
    echo "
".$row["description"]."
";

    // Tags
    $tags = explode(",", $row["tags"]);
    if (count($tags))
    {
        echo "
";
    foreach ($tags as $tag)
        {
            echo "".$tag."";
    }
        echo "
";
    }

    // Colours
    $colours = explode(",", $row["colours"]);
    if (count($colours))
    {
        echo "

Colours
";
    foreach ($colours as $colour)
    {
            echo "

";
    }
    echo "
";
    }

    // Services
    $services = explode(",", $row["services"]);
    if (count($services))
    {
        echo "

Services

";
    foreach ($services as $service)
    {
            echo "
".$service."
";
    }
    echo "

";
    }
    echo "
Visit Website
";
    echo "
";
    echo "

";
    echo "
";
    echo "

";
    echo "

";
    echo "

";
    echo "
";
}

?>

The Client List Page

$getPortofolioDataSql = "SELECT `id`, `title`, `description`, `client` FROM `client`";
$statement = $databaseLink->prepare($getPortofolioDataSql);
$statement->execute();
if($statement->rowCount() == 0)
{
    echo "No Portfolio Found";
}
else
{
    while($portofolioDetails = $statement->fetch(PDO::FETCH_ASSOC))
    {
        echo "
";
    echo "";
        echo "
";
    }
}
?>

Your database queries are using bound parameters with prepared statements. Good.
You need to escape your output.

Other observations:

Your Database class adds nothing (unless you've abbreviated it here for simplicity?). Consider simply using PDO.
You have no error handling. Try loading your page with incorrect db credentials and see what I mean.
Consider using markup with php instead of string-ified markup in php. It is much easier to read and maintain.


Is the use of Struct or Class personal preference? I have a solution in which I instantiate a class with various types inside of it.  I feel it is becoming to cumbersome to instantiate and then create  variables for each of the types I want to access in the class (Option 2 in the code), so I've come up with a custom return type so I only have to return data once for use (option 1).

So with the struct my code is more readible in the MAIN block, but is there a better way to do this?

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

namespace retclass
{
    class Program
    {
        static void Main(string[] args)
        {

            //Option 1 Struct and function
            //create struct
            ret x = new ret();
            x = goofie(3, "this is the struct string");
            Console.WriteLine("The String is {0}, the int is {1} and the value is {2}", x.str, x.num, x.yesno);

            //Option 2 Class
            //Instantiate retclass
            retclass retins = new retclass();
            retins.num = 3;
            retins.str = "this is the class string";
            if ((retins.num > 0) && retins.str != "")
            {
                retins.yesno = true;
            }
            Console.WriteLine("The String is {0}, the int is {1} and the value is {2}", retins.str, retins.num, retins.yesno);
            Console.ReadLine();

        }

        public struct ret
        {
            public int num { get; set; }
            public string str { get; set; }
            public bool yesno { get; set; }
        }

        public static ret goofie(int isn, string ss)
        {
            ret dex = new ret();
            dex.str = ss;
            dex.num = isn;
            if ((dex.str != "") && (dex.num > 0))
            {
                dex.yesno = true;
            }
            return dex;
        }
    }

    public class retclass
    {
        public int num { get; set; }
        public string str { get; set; }
        public bool yesno { get; set; }
    }

}

This MSDN Page states:

  CONSIDER defining a struct instead of a class

  if instances of the type are small and commonly short-lived or are commonly embedded in other objects.

  AVOID defining a struct unless the type has all of the following characteristics:

  It logically represents a single value, similar to primitive types (int, double, etc.).
  It has an instance size under 16 bytes.
  It is immutable.
  It will not have to be boxed frequently.

So if you want to use a class instead of a structure, you can just add the same static goofy method, to get the same style like you get with the structure:

   public static retclass goofie(int isn, string ss)
    {
        retclass dex = new retclass();
        dex.str = ss;
        dex.num = isn;
        if ((dex.str != "") && (dex.num > 0))
        {
            dex.yesno = true;
        }
        return dex;
    }

You can either add this to the Main or better inside the class itself. If you consider the second, you can call it like:

retclass x = retclass.goofie(3, "this is the reclass string");

Console.WriteLine("The String is {0}, the int is {1} and the value
      is {2}", x.str, x.num, x.yesno);


Stack implementation in python using jupyter This is my first program in Python: a stack implementation.

class Stack:
    items =[];
    index =-1;

    def isEmpty(self):
        return  self.index >=0;

    def push(self,item):
            self.index  +=1;
            print(self.index);
            self.items.insert(self.index,item);

    def pop(self):

        if self.index<0:
            raise Exception(" no items in list");
        else:
            del self.items[self.index];
            self.index -=1;

    def peek(self):
        print(self.index);
        if self.index >-1:
            return self.items[self.index];

stackobj = Stack();
stackobj.push("solomon");
stackobj.push("helloworld")
stackobj.pop();

print(stackobj.peek())

Even though this program works I'm still confused as to whether I've written the program in the right way. I am also not sure when to use the self operator.

Welcome to the land of Python!

If you come from Java, think as self being the same as this in Java.
It is a reference to the object, the difference being in Python, you have to explicitly pass self as a parameter to the class methods.

If you write in Java

class Stack{

    //constructor and class variables goes here

    void pop(){
       if(this.index < 0)
       //logic of the function goes here
    }
}

In Python you write like you did here

class Stack:
    # constructor and class variables goes here

    def pop(self):
        if self.index<0:
        # logic of the function goes here

You see that self and this serve the same purpose.

An another point, you should use documentation! As in PEP257 the format recommanded is like:

def complex(real=0.0, imag=0.0):
    """Form a complex number.

    Keyword arguments:
    real -- the real part (default 0.0)
    imag -- the imaginary part (default 0.0)
    """
    if imag == 0.0 and real == 0.0:
        return complex_zero
    ...
        '''

Another thing, to follow PEP8 it is better to write assignment and condition with a space after the operator.
Write  items = [] or self.index >= 0 instead of items =[] or self.index >=0.

Also you can drop ; in Python, it's not Java :)

Edit: Also look at @Jon's comments below


Combining 3 or more arrays in php Assuming I have 5 arrays, all just indexed arrays, and I would like to combine them, this is the best way I can figure, has anyone found a more efficient solution?

function mymap_arrays() {
    $args = func_get_args();
    $key  = array_shift($args);

    return array_combine($key, $args);
}

$keys   = array('u1', 'u2', 'u3');
$names  = array('Bob', 'Fred', 'Joe');
$emails = array('bob@mail.com', 'fred@mail.com', 'joe@mail.com');
$ids    = array(1, 2, 3);

$u_keys = array_fill( 0, count($names), array('name', 'email', 'id') );

$users  = array_combine($keys,
            array_map('mymap_arrays', $u_keys, $names, $emails, $ids)
          );

This returns:

Array
(
    [u1] => Array
        (
            [name] => Bob
            [email] => bob@mail.com
            [id] => 1
        )

    [u2] => Array
        (
            [name] => Fred
            [email] => fred@mail.com
            [id] => 2
        )

    [u3] => Array
        (
            [name] => Joe
            [email] => joe@mail.com
            [id] => 3
        )

)

EDIT: After a lot of benchmarking, this is the fastest solution I've come up with

function test_my_new() {
    $args    = func_get_args();
    $keys    = array_shift($args);
    $vkeys   = array_shift($args);
    $results = array();

    foreach($args as $key => $array) {
        $vkey = array_shift($vkeys);

        foreach($array as $akey => $val) {
            $result[ $keys[$akey] ][$vkey] = $val;
        }
    }

    return $result;
}

$keys   = array('u1', 'u2', 'u3');
$names  = array('Bob', 'Fred', 'Joe');
$emails = array('bob@mail.com', 'fred@mail.com', 'joe@mail.com');
$ids    = array(1,2,3);
$vkeys  = array('name', 'email', 'id');

test_my_new($keys, $vkeys, $names, $emails, $ids);

I would suggest the following:

function combine_keys_with_arrays($keys, $arrays) {
    $results = array();

    foreach ($arrays as $subKey => $arr)
    {
       foreach ($keys as $index => $key)
       {
           $results[$key][$subKey] = $arr[$index];
       }
    }

    return $results;
}

$keys    = array('u1', 'u2', 'u3');
$names   = array('Bob', 'Fred', 'Joe');
$emails  = array('bob@mail.com', 'fred@mail.com', 'joe@mail.com');
$ids     = array(1,2,3);

combine_keys_with_arrays($keys, array('name'  => $names,
                                      'email' => $emails,
                                      'id'    => $ids));

The second parameter to the function can be built as you need it.  It avoids array shifting and variable arguments, but I think it should be pretty fast.


Python log entry search utility I wrote a small utility for searching our web app log files for certain strings. The idea is that if the search string is found somewhere in the entry, the entire entry should be printed.

The constraints are:

One log entry can have one to many lines (stacktraces for example)
Each log entry starts with an uppercase string specifying log type, followed by a pipe character ("ERROR|...")
Each log entry ends with a pipe character followed by an exclamation mark ("|!")
The utility should work much like a regular grep command, meaning that it takes input either from a list of files or from STDIN.

The script works as expected, but it is quite slow. Writing the same logic in an AWK script is about twice as fast when processing a large amount of data. Any suggestions on how to improve the performance?

logentrygrep.py

#!/bin/python2.7

from __future__ import print_function
import argparse
import fileinput
import re
import signal
import sys

# Disable printing stacktrace on KeyboardInterrupt
signal.signal(signal.SIGINT, lambda x,y: sys.exit(1))

# Parse arguments
parser = argparse.ArgumentParser(description='Tool for greping log entry contents')
parser.add_argument('pattern', help='Pattern to search for')
parser.add_argument('file', nargs='*', help='Files to search')
parser.add_argument('-i', '--ignore-case', dest="ignorecase", action='store_true', help='Case insensitive greping')
parser.add_argument('-v', '--verbose', dest="verbose", action='store_true', help='Print verbose messages')
args = parser.parse_args()

VERBOSE = args.verbose

if VERBOSE:
    print("Pattern to search for: " + args.pattern)
    print("File(s): " + (str(args.file) if args.file else "STDIN"))

class LogDataProcessor:
    entryHeadPattern = re.compile("^[A-Z]{3,}\|")
    entryTailPattern = re.compile("\|!$")

    totalLinesProcessed = 0
    lineBuffer = []
    matchFound = False

    def __init__(self, regex_pattern, ignorecase):
        if ignorecase:
            self.pattern = re.compile(regex_pattern, re.IGNORECASE)
        else:
            self.pattern = re.compile(regex_pattern)

    def process(self, indata):
        for line in fileinput.input(indata):
            self.processLine(line.rstrip())

    def processLine(self, line):
        self.totalLinesProcessed += 1

        if self.isEntryHead(line):
            # Flush in case previous entry did not terminate correctly
            self.flush()
            self.addToBuffer(line)

            # If it's a one-line entry (head and tail on the same line)
            if self.isEntryTail(line):
                self.flush()

        elif not self.isBufferEmpty():
            self.addToBuffer(line)

            if self.isEntryTail(line):
                self.flush()

    def flush(self):
        if self.matchFound:
            self.printBuffer()

        self.lineBuffer = []
        self.matchFound = False

    def printBuffer(self):
        for line in self.lineBuffer:
            print(line)

    def addToBuffer(self, line):
        if not self.matchFound and self.lineMatches(line):
            self.matchFound = True

        self.lineBuffer.append(line)

    def isBufferEmpty(self):
        return not self.lineBuffer

    def isEntryHead(self, line):
        return self.entryHeadPattern.match(line)

    def isEntryTail(self, line):
        return self.entryTailPattern.search(line)

    def lineMatches(self, line):
        return self.pattern.search(line)

processor = LogDataProcessor(args.pattern, args.ignorecase)
processor.process(args.file)

if VERBOSE:
    print("Lines searched: " + str(processor.totalLinesProcessed))

On my windows 10 laptop, I ran your program on a dummy log file under the python
profiler using the command line:

python -m cProfile -s cumulative loggrep.py "speech" \data\test.log

The dummy log file has about 4.3 Mbytes of text spread across about 100k lines
comprising 32772 log entries.  The search pattern was "speech", which occured
444 times.

This is the ouput of the profiler:

     1322924 function calls (1322872 primitive calls) in 2.006 seconds

Ordered by: cumulative time

ncalls  tottime  percall  cumtime  percall filename:lineno(function)
   7/1    0.000    0.000    2.006    2.006 {built-in method builtins.exec}
     1    0.000    0.000    2.006    2.006 loggrep.py:3(<module>)
     1    0.077    0.077    1.997    1.997 loggrep.py:38(process)
100003    0.136    0.000    1.788    0.000 loggrep.py:42(processLine)
 32772    0.013    0.000    1.369    0.000 loggrep.py:60(flush)              <-----
   444    0.014    0.000    1.355    0.003 loggrep.py:67(printBuffer)        <-----
  9722    1.341    0.000    1.341    0.000 {built-in method builtins.print}  <-----
100003    0.061    0.000    0.132    0.000 loggrep.py:71(addToBuffer)
100004    0.042    0.000    0.116    0.000 fileinput.py:248(__next__)
100003    0.028    0.000    0.075    0.000 loggrep.py:80(isEntryHead)
100004    0.060    0.000    0.071    0.000 {method 'readline' of '_io.TextIOWrapper' objects}
195293    0.068    0.000    0.068    0.000 {method 'search' of '_sre.SRE_Pattern' objects}
100003    0.027    0.000    0.062    0.000 loggrep.py:83(isEntryTail)
 95290    0.027    0.000    0.060    0.000 loggrep.py:86(lineMatches)
100010    0.047    0.000    0.047    0.000 {method 'match' of '_sre.SRE_Pattern' objects}

The header shows the program took 2.006 seconds to run. Looking at the
lines marked with "<-----", we see that flush() was called 32772 times
and took about 1.369 seconds.  From the code, we know flush() calls
printbuffer(), which calls print().  The profile says print() was
called 9722 times taking 1.341 seconds, or about 65% of the total run time.

I changed entryTailPattern to "\|!\s*$".  This let me get rid of the
.rstrip() on each line so that the lines would keep the '\n' at the end.
This let me change the print() for loop in printBuffer() to a call to
sys.stdout.writelines().  This saved about 0.5 seconds over the print()
for loop.

All the other function calls seem to be in the noise, so i don't see any other
promising things to try.


Given a string, return a string where every character in the original is doubled For example, given the string "xyz", return the string "xxyyzz".

I was given this as a part of a test. I would really appreciate if you can help me to find a better way of doing it.

I came up with two methods to do the same thing but one was an extension method.

I have a couple more questions which I will post in separate posts.

public class RepeatCharactersInString
{
    public string RepeatAString(string sInputString, int repeatCount)
    {

        StringBuilder sOutputString = new StringBuilder();
        if (string.IsNullOrEmpty(sInputString))
        {
            Console.WriteLine("Empty string !");
        }
        else
        {
            foreach (char c in sInputString)
            {
                sOutputString.Append(new String(c, repeatCount));
            }
        }

        return sOutputString.ToString();
    }

}

// Method 2 Using extension methods
public static class RepeatCharactersInStingExtensions
{

    public static string RepeatAllCharactersInThisString(this string sInputString, int repeatCount)
    {
        StringBuilder sOutputString = new StringBuilder();
        if (string.IsNullOrEmpty(sInputString))
        {
            Console.WriteLine("Empty string !");
        }
        else
        {
            foreach (char c in sInputString)
            {
                sOutputString.Append(new String(c, repeatCount));
            }
        }

        return sOutputString.ToString();
    }
}

No reason to write anything when string is empty.

public string RepeatAString(string sInputString, int repeatCount)
{
    StringBuilder sOutputString = new StringBuilder();

    foreach (char c in sInputString)
        sOutputString.Append(new String(c, repeatCount));

    return sOutputString.ToString();
}

I think your solution is the best. Obviously you can use LINQ if you want but IMO it doesn't bring anything to table.

public string RepeatAString(string input, int count)
{
    return String.Concat(input.Select(c => new String(c, count)));

}


Staircase problem solved using recursion Problem
Question taken from here.

I want to go up a flight of stairs that has n steps. I can either take 1 or 2 steps each time. How many different ways can I go up this flight of stairs? Write a function count_stair_ways that solves this problem for me.
def count_stair_ways(n):

Solution:
def count_stair_ways(n):
    if n == 1: # 1 way to flight 1 stair case step
        return 1
    if n ==2: # 2 ways to flight 2 stair case steps(1+1, 2)
        return 2
    return count_stair_ways(n-1) + count_stair_ways(n-2)

Can we improve this code?

The staircase problem actually just generates the Fibonnacci sequence.

Whilst the recursive solution is nice, without memoization you're much better off just using a loop:

def count_stairways(n):

   a, b = 0, 1
   for _ in range(n):
       a, b = b, a+b

   return b

A nice alternative if you want multiple values out is to create a generator:

def count_stairways():

    a, b = 0, 1
    while True:
        a, b = b, a+b
        yield b

If you just want a particular value (possibly for a large n), the fibonnacci sequence actually has a closed form

 def count_stairways(n):

     phi = (1 + math.sqrt(5)) / 2

     return int((pow(phi, n+1) - pow(1-phi, n+1))/math.sqrt(5))


Generic Object Pool in C# I have a limit of 100 sessions (each session held in an object). Many threads will be requesting sessions, and it is very possible that all 100 sessions could be in use at any time.

I'm a Junior Developer. I saw the Object Pool pattern and decided that it looked appropriate. What do you think? If no sessions are avaible, the thread that requested it will get an Exception back and have to try again.

I'm using IOC with ISessionPool as a singleton. In testing it seems to work well, but I'd appreciate any feedback.

public class SessionPool<T> : ISessionPool<T>
{
    private readonly List<T> _inUse;
    private readonly List<T> _available;
    private readonly Action<T> _cleanerAction;

    /// <summary>
    /// Creates an object pool for objects of type T.
    /// NOTE: All availbale instances are initialised immediately.
    /// </summary>
    /// <param name="instanceCreator">A function that creates objects of type T</param>
    /// <param name="maxInstances">The maximum number of instances</param>
    /// <param name="cleanerAction">An action that cleans up the object before it is returned to the pool</param>
    public SessionPool(Func<T> instanceCreator, int maxInstances, Action<T> cleanerAction)
    {
        _inUse = new List<T>(maxInstances);
        _available = new List<T>(maxInstances);
        _cleanerAction = cleanerAction;

        CreateAllInstances(instanceCreator, maxInstances);
    }

    private void CreateAllInstances(Func<T> instanceCreator, int numberOfInstances)
    {
        for (int i = 0; i < numberOfInstances; i++)
        {
            _available.Add(instanceCreator.Invoke());
        }
    }

    public T GetSession()
    {
        lock (_available)
        {
            if (_available.Count == 0)
            {
                throw new NoAvailableSessionsException("Unable to get session");
            }
            else
            {
                T toReturn = _available[0];
                _inUse.Add(toReturn);
                _available.RemoveAt(0);
                return toReturn;
            }
        }
    }

    public void ReleaseSession(T session)
    {
        // Clean up the object before making it available again
        _cleanerAction.Invoke(session);

        lock (_available)
        {
            _available.Add(session);
            _inUse.Remove(session);
        }
    }

    public int GetAvailableSessionCount()
    {
        lock (_available)
        {
            return _available.Count;
        }
    }
}

A security problem is ReleaseSession adds whatever T session passed to _available, whether it was issued by you or not.

You can change _inUse like so:

HashSet _inUse = new HashSet<T>(new ObjectReferenceEqualityComparer<T>())

where ObjectReferenceEqualityComparer is an EqualityComparer that uses Object.ReferenceEquals and check whether _inUse contains session before adding it to the _available to avoid that problem.

You may think because you intend, for now, to use this pool with class X, which you think, for now, this problem does not apply. It may turn out that this problem applies to X after all, or someone else, or even yourself, may use this pool for some other class Y.


Temperature Interpolation I want to interpolate temperature in correlation with height. I have temperature data from stations with coordinates and height in this format:

Lat     Lon     Temp     Height

46.2956 16.3978 10.71196 120

45.9776 16.0159 11.78539 150

45.8521 15.9598 11.18838 133

etc.

After reading the data there are 4 arrays (y, x, temp, and height) and variable z which represents correlation between temperature and height, e.g., z = -0.005 which means that for each meter of height temperature decreases for 0.005 °C.

Z is calculated like this: z = np.polyfit(height, temp, 1,2)

After that I can calculate temperature on 0 meters for each station:

for i in range(len(temp)):
            temp0m.append((temp[i])-(height[i]*z))

I also have data with heights in meters with resolution of 0.003333333333 degrees and I want to use these heights and interpolated temperature to create map.

Heights are in "dem_final2.txt" (ncolums = 2700, nrows = 1560)

1653 1571 1493 1429 1354...
1730 1699 1620 1528 1399...

So this is what I've done so far, everything is working but it's very slow:

if __name__ == '__main__':
    nx, ny = 1080,624
    chunkSize = 10
    t = []
    grid2 = []
    xEast = 12
    ySouth = 42
    xWest = 21
    yNorth = 47.2
    res = 0.003333333333
    x,y,temp0m,z = downloadData()

    xi = np.linspace(xEast+0.1, xWest-0.1, nx)
    yi = np.linspace(ySouth+0.1, yNorth-0.1, ny)
    xi, yi = np.meshgrid(xi, yi)
    xi, yi = xi.flatten(), yi.flatten()

    l = len(xi)/chunkSize
    for i in range(chunkSize):
        interp = rbfInterp(x,y,temp0m,xi[(l*i):(l*(i+1))],yi[(l*i):(l*(i+1))])
        grid2 = grid2 + list(interpData(xi[(l*i):(l*(i+1))],yi[(l*i):(l*(i+1))],z,interp,xEast,ySouth,xWest,yNorth,res))[(l*i):(l*(i+1))]

    grid2 = np.array(grid2, np.float)
    grid2 = grid2.reshape((ny, nx))
    plot (x,y,temp0m,grid2) #plotting to png with matplotlib

def rbfInterp(x, y, temp0m,xi,yi):
    interp = Rbf(x, y, temp0m, function='linear')
    return interp(xi,yi)

def interpData(xi,yi,z,interp,xEast,ySouth,xWest,yNorth,res):
    append = t.append
    row= -1
    for i in xrange(len(xi)):
        tmp = int(round((yi[i]-ySouth)/res,0))
        tmp1 = int(round((xi[i]-xEast)/res,0)+1)
        if (row!=tmp):
            tmp2 = linecache.getline('dem_final2.txt', tmp).split(' ')
        if (float(tmp2[tmp1])>-10):
            append ((float(tmp2[tmp1]))*z+float(interp[i]))
        else:
            append (None)
        row = tmp
    return t

I divided into chunks because a lot of memory was used without them and I think this can be used to speed up things maybe with more processes or something like that but I don't have knowledge to do that without any help. Of course, any other help is also welcome.

Btw this script runs for about 25-30 sec

The problems start with this code being fairly unreadable. A few simple rules can help:

Every comma is followed by a space: xi, yi instead of xi,yi.
Every binary operator like + is surrounded by a space on each side:

int(round((xi[i] - xEast) / res, 0) + 1)

instead of int(round((xi[i]-xEast)/res,0)+1).
If an expression becomes to complicated, assign intermediate values to variables which also assist in explaining the code to a reader:

start = l * i
end = l * (i + 1)
chunk_x = xi[start:end]
chunk_y = yi[start:end]
interp = rbfInterp(x, y, temp0m, chunk_x, chunk_y)
result = interpData(chunk_x, chunk_y, z, interp, xEast, ySouth, xWest, yNorth, res)
grid2 += list(result)[start:end]

instead of

interp = rbfInterp(x,y,temp0m,xi[(l*i):(l*(i+1))],yi[(l*i):(l*(i+1))])
grid2 = grid2 + list(interpData(xi[(l*i):(l*(i+1))],yi[(l*i):(l*(i+1))],z,interp,xEast,ySouth,xWest,yNorth,res))[(l*i):(l*(i+1))]

At least try to provide meaningful names for your variables. Expressions like tmp2[tmp1] are unacceptable.
Avoid overly clever things like append = t.append; …; append(…), as you gain nothing by this indirection. t.append(…) directly.
In a function call, never put a space between the function name and the parens.

plot(x, y, temp0m, grid2) instead of plot (x,y,temp0m,grid2)
append(None) instead of append (None)

Most of Python is using the convention that variable names should use snake_case not camelCase. While consistency is more important than following any specific style, I suggest you adopt this common style.
Some variable names like chunkSize are misleading – this is the number of chunks, not their size.

Once we have cleaned up the code, we can investigate where performance could be increased. This is done mostly by avoiding unnecessary recalculations, but also by finding more efficient expressions.

For example, Rbf(x, y, temp0m, function='linear') will always produce the same value as the parameters never change. Therefore, calculate it once outside of any loops. Actually, you can remove the whole rbfInterp function.

Inside interpData, you translate and rescale the data: (yi[i]-ySouth)/res. If you do this outside of an explicit loop, then numpy can do this more efficiently: translated_yi = (yi - ySouth)/res.

The whole meshgrid business is obfuscating which calculation you are actually performing:

# prepare an interpolation function for this data
interpolation_function = Rbf(x, y, temp0m, function='linear')

# the actual coordinates
yis = np.linspace(ySouth + 0.1, yNorth - 0.1, ny)
xis = np.linspace(xEast + 0.1, xWest - 0.1, nx)

# the grid positions in the data file
yis_normalized = ((yis - ySouth)/res).round(0).astype(np.int)
xis_normalized = ((xis - xEast)/res + 1).round(0).astype(np.int)

grid = np.zeros((ny, nx))
for i in range(ny):
    heights = linecache.getline('dem_final2.txt', yis_normalized[i]).split(' ')
    for j in range(nx):
        height = float(height[xis_normalized[j]])
        result = height * z + interpolation_function(xis[j], yis[i])
        grid[i][j] = result if data_item > -10 else np.nan

Some important points:

We fetch and parse each line once per line, instead of once per grid column.
We factor out repeated calculations outside of the loops
We avoid any confusing reshaping by writing the results directly into the correct data structure.
We try to avoid explicit loops by using numpy transformations

The purpose of the code is now much more obvious.

However, the usage of linecache is quite suspicious. We read the lines in sequential order, so caching will be of no use → read the file manually. You should also consider reading it into a numpy array, as the file in this form should only be using a few MB of memory:

heights = np.loadtxt('dem_final2.txt')
...
result = heights[yis_normalized[i]][xis_normalized[j]] * z + ...


Project Euler #10 - summation of primes (From Project Euler)

  The sum of the primes below 10 is 2 + 3 + 5 + 7 = 17.

  Find the sum of all the primes below two million.

Can someone please help me improve this code?

import math

def main():
    lim = 2000000
    i = 2
    now = []
    total = 0
    for i in range(2,lim+1):
        for x in range(1,int(math.sqrt(i))+1):
            if len(now) ==2:
                break
            elif i % x == 0:
                now.append(x)

        if len(now) == 1:
            total += i
            print(total)
        now = []
    print(total)

First step : functions and tests

The very first thing you probably should write is a function computing what you really want : a sum of primes. Also, a value is provided as part of the problem, use this to write a test : as soon as something goes bad, you might notice it.

def sum_primes_to_limit(lim):
    i = 2
    now = []
    total = 0
    for i in range(2,lim+1):
        for x in range(1,int(math.sqrt(i))+1):
            if len(now) ==2:
                break
            elif i % x == 0:
                now.append(x)

        if len(now) == 1:
            total += i
        now = []
    return(total)

def main():
    assert sum_primes_to_limit(10) == 17
    print(sum_primes_to_limit(2000000))

Making things simpler

Declaring the i variable is not needed.
Similarly, the now variable does not need to be defined before the loop and cleared at the end of each iteration : just define it as empty at each iteration.

def sum_primes_to_limit(lim):
    total = 0
    for i in range(2,lim+1):
        now = []
        for x in range(1,int(math.sqrt(i))+1):
            if len(now) == 2:
                break
            elif i % x == 0:
                now.append(x)

        if len(now) == 1:
            total += i
    return(total)

Making things even simpler

You don't really care about the content of now, just its length. Also, there's no need to check that it does not get longer that 2 at each iteration : just check when you update the list.

Also, you can get rid of the list and make it a simple counter :

def sum_primes_to_limit(lim):
    total = 0
    for i in range(2,lim+1):
        nb_div = 0
        for x in range(1,int(math.sqrt(i))+1):
            if i % x == 0:
                nb_div += 1
                if nb_div == 2:
                    break

        if nb_div == 1:
            total += i
    return(total)

More function

Given the way you check if a number is prime or not, it probably make sense to define a function for this :

def is_prime(i):
    nb_div = 0
    for x in range(1,int(math.sqrt(i))+1):
        if i % x == 0:
            nb_div += 1
            if nb_div == 2:
                return False
    return nb_div == 1

def sum_primes_to_limit(lim):
    total = 0
    for i in range(2,lim+1):
        if is_prime(i):
            total += i
    return(total)

A bit of Python cool syntax

Among the cool things in Python are generator expressions. Using generator expressions and sum, you can write sum_primes_to_limit in a concise and fancy way :

def sum_primes_to_limit(lim):
    return sum(i for i in range(2,lim+1) if is_prime(i))

Rewriting the function

You can avoid considering 1 as a divisor if you simply start at 2 (then, you might need some special processing for 1 but I'll leave this to you).

Then your function becomes :

def is_prime(i):
    nb_div = 0
    for x in range(2,int(math.sqrt(i))+1):
        if i % x == 0:
            nb_div += 1
            if nb_div == 1:
                return False
    return nb_div == 0

Then, you don't need the number anymore :

def is_prime(i):
    for x in range(2,int(math.sqrt(i))+1):
        if i % x == 0:
            return False
    return True

And this looks like it really wants to be written with generator expression (and all or any this time).

def is_prime(i):
    return all(i % x for x in range(2,int(math.sqrt(i))+1))

(Please note that I took this chance to remove a != 0 as non-zero boolean will evaluate to True).

This is all pretty cool but we want speed!

You want to compute/check primes up to a certain limit. Good for you there is an algorithm just for this : the Sieve of Eratosthenes.

Here's an implementation :

def sieve(lim):
    primes = [True] * (lim+1)
    primes[0] = primes[1] = False
    for i in range(2, int(math.sqrt(lim))+1):
        if primes[i]:
            for j in range(i*i, lim+1, i):
                primes[j] = False
    return primes

def sum_primes_to_limit(lim):
    primes = sieve(lim)
    return sum(i for i in range(2,lim+1) if primes[i])

Final details

I am used to Python 3 but if you are using Python 2, range generates an actual list. When this is not required (and in your case, this is not required), you should be using xrange.


More succinct / ideal solution to Project Euler #22 "Using names.txt (right click and 'Save Link/Target As...'), a 46K text file containing over five-thousand first names, begin by sorting it into alphabetical order. Then working out the alphabetical value for each name, multiply this value by its alphabetical position in the list to obtain a name score.

For example, when the list is sorted into alphabetical order, COLIN, which is worth 3 + 15 + 12 + 9 + 14 = 53, is the 938th name in the list. So, COLIN would obtain a score of 938 × 53 = 49714.

What is the total of all the name scores in the file?"

I'm relatively new to clojure, and this is what I came up with:

(def names (sort (map (fn[x] (replace x #"\"" "")) (split (slurp "/users/calvinfroedge/Downloads/names.txt") #","))))

(loop [i 0 total 0]
  (if (not= i (count names))
    (recur (inc i) (+ total (* (inc i) (reduce + (map (fn[x] (- (int x) 64)) (nth names i))))))
    total)

I read somewhere that doseq is preferred to loop/recur, but it wasn't apparent to me how to comprehensibly AND idiomatically approach this problem without using an explicit loop with an incrementing value.

Am I missing something?

Yes. I think you still miss a very useful part of clojure - using its various collection types to model your solution. In your problem, you could use the fact that a string is a seq and a map acts as a function to calculate the cost of a word:

;; this map will act as a function from a name to its ordinal
(def name->position (zipmap names (map inc (range))))

;; and this map will act as a function from letter to its ordinal
(def ab-map (zipmap alphabet (map inc (range))))

Now that we have these two functions, we could easily calculate a name's value in this way:

(defn score [word]
  (* (reduce + (map ab-map word))
     (name->position word 0)))

You can now calculate COLIN:
(score "COLIN")

Explicit recursion can many times be avoided by using functions like map and reduce, mainly because often the sequences themselves are defined in terms of recursion

From here, you could solve this question by using map and reduce with the new function score


Hex dump of a list of 16-bit numbers This is some code which prints a hexdump of a list of 16 bit numbers, at 16 bytes distance for each, after the "print data" comment:

#!/usr/bin/python3

# creating test data
data = []
for i in range(500):
    data = data + [ i & 0xff, i >> 8, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 ]

# print data
c = 0
out = ""
for i in range(int(len(data) / 16)):
    if i == 512: break
    low = data[i * 16]
    high = data[i * 16 + 1]
    d = low | (high << 8)
    out = out + ("%04x " % d)
    if (i % 16) == 15: out = out + "\n"
print(out)

It works, but I think I can write it simpler with "join" and maybe list comprehension? But shouldn't be a cryptic one-liner or something, I'm just looking for a more idiomatic Python solution. I want to print the first 512 numbers, but the data array can be shorter or longer.

Here are some comments on your code:

I assume the structure of the input data is to be taken as it is: one long list where only the 1 out of 8 values contribute to the output.
c = 0 is defined but never used
It is not recommended to use compound statements (see PEP8 - "Other Recommendations"): use a separate line for an if and the statement under condition.
/ performs a true division (giving a float). To avoid switching between float and int, use integer division: //
if i == 512: break could me omitted if you would limit the range of the for loop immediately. Instead of len(data) use min(512*16, len(data))
The multiplication i * 16 can be avoided if you use the step argument of range() so that i takes multiples of 16.
Instead of "%04x " % d use the newer f-strings
Instead of calculating d, you could just pass high and low to the string template and format each independently.
Instead of if (i % 16) == 15: you could use a nested loop that deals with one output line
Your code produces a blank at the end of each line. That seems unnecessary. With " ".join you would not have this extra blank.
out has a terminating \n, but print also prints a newline as terminator (by default). With "\n".joinyou would not have this extra newline

Here is how it could look:

# Set a maximum to the output
length = min(512*16, len(data))

# Format data
lines = []
for line in range(0, length, 256):
    items = []
    for i in range(line, min(line+256, length), 16):
        items.append(f"{data[i+1]:02x}{data[i]:02x}")
    lines.append(" ".join(items))
out = "\n".join(lines)

# Output
print(out)

Here is how the above data formatting translates when using list comprehension. You can split the expression over multiple lines to improve readability:

# Format data
out = "\n".join([
            " ".join([
                f"{data[i+1]:02x}{data[i]:02x}"
                for i in range(line, min(line + 256, length), 16)
            ])
            for line in range(0, length, 256)
        ])


Finding corners from Line arrays I have the following code that takes input of Line arrays and attempts to find the intersections on the end points between them. The value _nConrerTolerance is an int of a value of around 10% of the average Line length to allow the lines to not need to be exactly on each other. Now obvisouly this code will be quite slow, especially as I have similar methods for top left, bottom left and bottom right corners. Can anyone suggest a way to improve this?

Also if you have any tips on how I can improve my coding style, if needed, that would be great.

private Point[] FindTopRightCorners(Line[] horizontalLines, Line[] verticalLines)
{
    List<Point> TopRightCorners = new List<Point>();

    Line[] laHorizontalLines = horizontalLines.OrderByDescending(l => l.EndPoint.X).ThenBy(l => l.EndPoint.Y).ToArray();
    Line[] laVerticalLines = verticalLines.OrderByDescending(l => l.StartPoint.X).ThenBy(l => l.StartPoint.Y).ToArray();

    foreach (Line verticalLine in laVerticalLines)
    {
        foreach (Line horizontalLine in laHorizontalLines)
        {
            if (verticalLine.StartPoint.X >= (horizontalLine.EndPoint.X - _nCornerTolerance) && verticalLine.StartPoint.X <= (horizontalLine.EndPoint.X + _nCornerTolerance))
            {
                if (horizontalLine.EndPoint.Y <= (verticalLine.StartPoint.Y + _nCornerTolerance) && horizontalLine.EndPoint.Y >= (verticalLine.StartPoint.Y - _nCornerTolerance))
                {
                    int nX = Math.Abs((verticalLine.StartPoint.X + horizontalLine.EndPoint.X) / 2);
                    int nY = Math.Abs((verticalLine.StartPoint.Y + horizontalLine.EndPoint.Y) / 2);

                    TopRightCorners.Add(new Point(nX, nY));
                    break;
                }
            }
        }
    }

    return TopRightCorners.Distinct().ToArray();
}

And the Line class is defined as follows:

public class Line
{
    public Point StartPoint { get; private set; }

    public Point EndPoint { get; private set; }

    public Line(Point startPoint, Point endPoint)
    {
        this.StartPoint = startPoint;
        this.EndPoint = endPoint;
    }
}

You have similar code and similar performance issue in your other question about points instead of lines.  I answered that one here. Some of my answers can apply here particularly as to the repeated use of .ToArray()?

If you skipped everything as arrays and went with IEnumerable<Line> or IEnumerable<Point> for inputs, output, and anything inbetween then you might see some performance benefit.  Might.  Don't know until you run it.  But sounds like you are talking about big collections.

If you try using IEnumerable and things are still slow, I'd try to use Parallel.For with a range partitioner.

That should address your major concern.  On a minor concern, the Line class is good enough and looks like something I once wrote.  I eventually switched from StartPoint and EndPoint to Point1 and Point2 since some readers may think StartPoint should somehow be 'less than' when compared to EndPoint or that it should be drawn like a vector that somehow starts at StartPoint and is then drawn to EndPoint.  To avoid such needless notions, I went with the generic Point1 and Point2.


Get all routes that start with rt_ The code reads all my routes and return to my AJAX request the names of the routes without duplicates and just those that have the "rt_" prefix.
class RouteController extends Controller {
    public function getRoutes()
    {

        //Get all the routes.
        $routes = Route::getRoutes();

        //Setting up the vars
        $array = [''];

        foreach ($routes as $route) {
            $condicional = substr($route->getName(), 0, 3);
            $found = FALSE;

           //Filtering the routes with rt_ prefix
            if ($condicional == 'rt_') {

                 //Filering duplicates
                foreach ($array as $item) {
                    if ($item == $route->getName()) {
                        $found = TRUE;

                    }

                }

                //Echo without duplicates
                if ($found == FALSE) {
                    $array[] = $route->getName();

                }
            }
        }

        return response()->json($array, 200);

    }

}

One way to simplify this would be to create a collection with collect() and then call getName() on each route using Collection::map(). Then use collection::filter() to filter out the routes that don't start with rt_ using the string helper method Str::startsWith()
class RouteController extends Controller {
    public function getRoutes()
    {
        // call getName on each route and then get unique values
        $uniqueRoutes = collect(Route::getRoutes())
            ->map(fn($route) => $route->getName())
            ->unique();

        //filter out any routes not starting with rt_
        $filteredRoutes = $uniqueRoutes->filter(function($name) {
             return Str::startsWith($name, 'rt_');
        });
        return response()->json($filteredRoutes, 200);

    }
}

Note - the uses the PHP 7.4 arrow function syntax could be used to simplify the callback to the filter() method
One could also use collection::reduce() to only add routes to the list that contain the desired prefix.

The existing code contains this block:

          //Filering duplicates
           foreach ($array as $item) {
               if ($item == $route->getName()) {
                   $found = TRUE;
               }
           }

That could be simplified using the PHP function in_array().

Another suggestion is to always use strict equality comparisons (i.e. === and !==) when possible - e.g. when $found should either be true or false then compare without the possibility of type conversion (i.e. with ==).


Binary Reflected Gray Code generation recursively I tried to write BRGC recursively. I've really recognized that intrinsic of recursion helps for some problems in which I think BRGC is included. is the way I wrote it recursively adequate solution? Can it be improved w.r.t recursion?

Do both methods have same complexity Theta(n), right?
T(n) = T(n-1) + n, T(n) = T(n/2) + n respectively.

import java.util.ArrayList;
import java.util.List;

class Test {
    public static void main(String[] args) {
        System.out.println(genGray(List.of("0", "1"), 3));
        System.out.println(genGrayV2(List.of("0", "1"), (int)Math.pow(2, 3)));
    }

    public static List<String> genGray(List<String> list, int n) {
        if (list.size() == (int)Math.pow(2, n+1))
            return list;

        List<String> combinedList = new ArrayList<>();

        for (String elem : list) {
            combinedList.add("0" + elem);
        }

        for (String elem : list) {
            combinedList.add("1" + elem);
        }

        return genGray(combinedList, n);
    }

    public static List<String> genGrayV2(List<String> list, int n) {
        if (n == 1)
            return list;

        List<String> combinedList = new ArrayList<>();

        for (String elem : list) {
            combinedList.add("0" + elem);
        }

        for (String elem : list) {
            combinedList.add("1" + elem);
        }

        return genGray(combinedList, n / 2);
    }
}

Not Gray Code

The first big issue with this code that I see is naming things. Naming a list list adds absolutely no semantic information. You're duplicating what the "syntax" already tells you. n also doesn't tell us anything. Let's try the following:

public static List<String> generateGray(List<String> currentGrayWords, int targetBitLength) {

public static List<String> generateGrayV2(List<String> currentGrayWords, int wordsToGenerate) {

Note that the semantics of these two methods are not interchangeable. There's quite a few assumptions you make about how parameters are passed in. The first assumption is that you think that the "currentGrayWords" are a valid and complete BRGC (and non-empty). The second assumption is that n is a clean power of two in the first version and positive in both versions.

Check your assumptions. Consider the following invocations that look completely harmless but generate totally unintended results:

generateGray(Collections.emptyList(), 8); // returns empty list
generateGray(Arrays.asList("0", "01", "10", "1"), 4); // returns inconsistent gray code
generateGray(Arrays.asList("0", "1"), 5); // stackoverflow because recursion never terminates
generateGrayV2(Arrays.asList("000", "001", "011", "010", "110", "111", "101", "100"), -4);
// stackoverflow, because it never terminates

Gray Code

As it stands your code isn't even correct. Wikipedia tells me that a binary reflected gray code works by concatenating the original list with each element having 0 prepended and the reflected original list with each element having 1 prepended. That reflection is conspiciously absent in the code you present here.

As such the following is what your code creates:

genGray(Collections.singletonList(""), 8);
"0" "1"
"00" "01" "10" "11" // gray-code property violated
"000" "001" "010" "011" "100" "101" "110" "111"

The correct sequence would've been:

"0" "1"
"00" "01" "11" "10"
"000" "001" "011" "010" "110" "111" "101" "100"

Method

What this code is missing is a comprehensive test-suite that actually checks inputs that you didn't consider when first writing this code. You may want to not expose the List<String> as input parameter. It could (should?) be something internal to what the code does.

Try to break your code! If you don't succeed, try harder. Throw things at it that "nobody would ever think of doing". That way when somebody actually does, the code handles it gracefully.

With recursion it's especially important to think of the "termination condition" in your code. Make sure whatever is thrown into the method, it always terminates. Stack Overflows are not an acceptable result ...


Simple Math Quiz Game I have been studying C# for 2 weeks already, I wrote a simple math game but not quite sure how bad it is. I'm just teaching myself to code through internet references. If you could review my work, I would appreciate it.

It is a simple math quiz game
player gets 1 point in every correct answer
the game will be over when the player gives the incorrect answer
it displays total score the player earned, then the score resets.
Is my program reasonable or is there anything i need to improve?

    static void Main(string[] args)
    {
        Console.Write("Please enter your Name: ");
        string userName = (Console.ReadLine());

        Console.WriteLine("Hello " + userName + ", Press ENTER to start the Math Quiz");

        Console.WriteLine();
        Console.ReadKey();

        Start:

        Random numberGenerator = new Random();
        int score = 0;

        while (true)
        {
            int num01 = numberGenerator.Next(1, 11);
            int num02 = numberGenerator.Next(1, 11);

            Console.WriteLine("What is " + num01 + " times " + num02 + " equal to?");

            int Answer = Convert.ToInt32(Console.ReadLine());
            int correctAnswer = num01 * num02;

            if (Answer == num01 * num02)
            {
                Console.ForegroundColor = ConsoleColor.Green;
                ++score;
                int responseIndex = numberGenerator.Next(1, 5);
                switch (responseIndex)
                {
                    case 1:
                        Console.WriteLine("Great!");
                        Console.WriteLine("Your score: " + score);
                        break;
                    case 2:
                        Console.WriteLine("You nailed it!");
                        Console.WriteLine("Your score: " + score);
                        break;
                    case 3:
                        Console.WriteLine("You're correct!");
                        Console.WriteLine("Your score: " + score);
                        break;
                    default:
                        Console.WriteLine("Good Job " + userName + ", Keep it up!");
                        Console.WriteLine("Your score: " + score);
                        break;
                }
                Console.ResetColor();
                Console.ReadLine();
                Console.WriteLine();
            }
            else
            {
                Console.ForegroundColor = ConsoleColor.Red;
                int responseIndex2 = numberGenerator.Next(1, 5);
                switch (responseIndex2)
                {
                    case 1:
                        Console.WriteLine("Are you even trying? The correct answer is " + correctAnswer);
                        break;
                    case 2:
                        Console.WriteLine("Ooops!!! The correct answer is " + correctAnswer);
                        break;
                    case 3:
                        Console.WriteLine("Oh, come on " + userName + " I know you can do better than that! The correct answer is " + correctAnswer);
                        break;
                    default:
                        Console.WriteLine("Sorry " + userName + ", that's incorrect, the correct answer is " + correctAnswer);
                        break;
                }
                Console.WriteLine(Environment.NewLine + "Game Over, Your score: " + score);
                Console.ResetColor();
                Console.WriteLine();
                Console.ReadLine();
                goto Start;
            }
        }
    }

Even though this is a fairly short program, resist the temptation to do everything in the Program class of your console app. Whenever I start a new console app, the first thing I do is create a Runner class with an Execute() method and call that from the Main() of Program.

Using goto is IMHO incredibly rare in a C#/.NET program. Make this code block a method and call it.

Whenever you start copy-pasting code, it's a sign you're doing something wrong.

switch (responseIndex)
{
    case 1:
        Console.WriteLine("Great!");
        Console.WriteLine("Your score: " + score);
        break;
    case 2:
        Console.WriteLine("You nailed it!");
        Console.WriteLine("Your score: " + score);
        break;
    case 3:
        Console.WriteLine("You're correct!");
        Console.WriteLine("Your score: " + score);
        break;
    default:
        Console.WriteLine("Good Job " + userName + ", Keep it up!");
        Console.WriteLine("Your score: " + score);
        break;
}

Clearly Console.WriteLine("Your score: " + score); should be in a method of its own, and the other line you be as well, with its message the parameter for that method, e.g.

private void ReportResult(string message, int score)
{
    Console.WriteLine(message);
    Console.WriteLine("Your score: " + score);
}

You could even write the second line as Console.WriteLine("Your score: {0}", score); or Console.WriteLine($"Your score: {score}");. While a small amount of string concatenation isn't a performance killer, newer versions of C# offer nicer and more readable ways to format strings.

int Answer: "Answer" should be camelCase.

Considering that you calculate int correctAnswer = num01 * num02;, why then do you do if (Answer == num01 * num02) next? Use correctAnswer.

Even though the current code is fairly short, I'd still move the code in the if and the code in the else block to a method of their own (e.g. "ReportSuccess" and "ReportFailure"). This will make your code easier to follow and splits up the logic into smaller chunks.


std::vector with custom growth factor So, yes, I know that "you shouldn't derive from std containers" but by now it's more of philosophical rule in my mind than a technical one. I've googled again for the one fundamental reason one should never-ever-ever-ever-do-that-ever-ever-ever-or-you-die but couldn't find it.

So, if there's nothing fundamentally wrong with the following and if we set aside performance considerations, can I a get a plain-old code review of this:

namespace growth
{
    template<size_t N>
    class linear
    {
    public:
        static_assert(N != 0, "Linear growth requires positive factor");

        static size_t grow(size_t size, size_t capacity)
        {
            return (size == capacity) ? capacity + N : capacity;
        }
    };

    template<typename Ratio = std::ratio<2, 1>>
    class geometric
    {
    public:
        static_assert(Ratio::num > Ratio::den, "Geometric growth requires ratio greater than 1");

        static size_t grow(size_t size, size_t capacity)
        {
            return (size == capacity) ? std::max((size_t)(capacity * (Ratio::num / (double)Ratio::den)), capacity + 1) : capacity;
        }
    };
}

template<typename T, typename Growth, class Allocator = std::allocator<T>>
class custom_growth_vector : public std::vector<T, Allocator>
{
public:
    using vector = std::vector<T, Allocator>;

    using vector::vector;

    void push_back(T&& value)
    {
        vector::reserve(Growth::grow(vector::size(), vector::capacity()));
        vector::push_back(value);
    }

    // Also replace emplace_back, insert, etc.
};

// ...

// Grows by 50% rather than double when size == capacity.
custom_growth_vector<int, growth::geometric<ratio<3, 2>>> cgvi;

Let's start with an attempt at a short summary of the situation with inheritance.

As long as you just create/use/destroy an object of your type, the fact that it inherits publicly from a standard container doesn't lead to a problem.

The problem arises if you have pointer (or reference) to the base class that refers to an object of the derived class, and you destroy the object via that pointer to base.

std::vector<int> *t = new custom_growth_vector<int, growth::geometric<ratio<3, 2>>>;

// ...

delete t; // This causes undefined behavior

Unfortunately, there's nothing you can do in your class to prevent this--part of the definition of public inheritance is that it allows implicit conversion to the base class.

Aside from that:

Linear growth

Using your growth::linear class means that any vector you create this way no longer meets the requirement for push_back to have amortized-constant complexity. In the other direction, this does mean the amount of space that's "wasted" by growing the vector is limited to a constant.

push_back

push_back normally has overloads for both const lvalue reference, and rvalue reference, though this may just be another member function you haven't provided (yet).

Growth Factor

I guess the biggest problem I see in the end is the fairly simple question what the real point of this is supposed to be. Most systems use virtual memory, which typically means that even if you think you're expanding an allocation following a geometric series, chances good that the system will allocate actual storage for your data following an arithmetic series, almost regardless of what you try to do.


Email confirmation model Email<input type="text" id="email" /><br />
Re-enter Email<input type="text" id="confirm-email" />

I'm thinking that I want a model so that I can have this partial page strongly typed.  But what a lame model....

It would look like this, and if this is a good Idea, what should I name this model?

public class ConfirmEmailModel?????
{
    [Required]
    [Display(Name = "Email",ResourceType=typeof(mvc3test.Resources.Common))]
    public string Email{ get; set; }

    [Display(Name = "Confirm Email")]
    [Compare("Email", ErrorMessage = "no match.")]
    public string ConfirmEmail{ get; set; }
}

call it a "view model"  as its not really a model,  its just a model for the view you are showing.

ConfirmEmailViewModel  :)


Make pinging faster while looking for a Windows PC This code pings 1 time every IP while using TTL to identify if it's a Windows PC. If so, it outputs it. It's very slow, so is there any way to make it go faster (1-255 in 12 min)?

@echo off
echo running, it will take a long time to scan
(
  FOR /L %%N in (1 1 254) DO (
      FOR /f "tokens=1,3 delims=: " %%A IN ('ping -n 1 192.168.250.%%N ^| find "TTL=128"') DO ECHO %%B

  )
) >ip.txt

echo done
get_info.bat

You will have to reduce the timeout interval with the -w option (milliseconds), but I don't know what is the minimum timeout that still gives reliable results.

You could try

ping -w 500 -n 1 192.168.250.%%N


VBA Script to Remove Duplicates I'm curious if there's any way I can substantially improve its speed (it runs through >200k rows). I think I'll most likely need this once, so please bear with me since it probably looks terrible.

Thanks for reading!

Code I would like reviewed:

Option Explicit

Public Sub RemoveDuplicates()
    Dim r As Range
    Dim rBefore As Range
    Dim wsSource As Worksheet
    Dim ws As Worksheet
    Dim c As Range
    Dim c_ As Range
    Dim v As Variant
    Dim v_ As Variant
    Dim dupCount As Long
    Dim xlApp As Application
    Dim i As Long
    Dim size As Long

    Set xlApp = Excel.Application
    With xlApp
        .ScreenUpdating = False
        .DisplayStatusBar = True
        .StatusBar = "Running Script..."
    End With

    Set wsSource = xlApp.ThisWorkbook.Worksheets("source")
    Set rBefore = wsSource.Range(wsSource.Cells(2, 1), wsSource.Cells(wsSource.UsedRange.Rows.Count, 1))

    Set ws = xlApp.ThisWorkbook.Worksheets("testSheet")
    Set r = ws.Range(ws.Cells(2, 1), ws.Cells(ws.UsedRange.Rows.Count, 1))
    size = r.Count
    i = 1
    For Each v In r
        Set c = v

        DoEvents
        dupCount = 0
        For Each v_ In r
            Set c_ = v_

            DoEvents
            If CStr(c.Value) = CStr(c_.Value) Then
                dupCount = dupCount + 1
                If dupCount > 1 Then
                    c_.Rows(c_.Row).EntireRow.Delete
                End If
            End If
        Next
        xlApp.StatusBar = Format(i & " out of " & size) & " Rows Complete"
        i = i + 1
    Next

    With xlApp
        .ScreenUpdating = True
        .DisplayStatusBar = False
        .DisplayAlerts = False
        .ThisWorkbook.Save
        .DisplayAlerts = True
        .ThisWorkbook.Close
    End With
End Sub

Seems Range.RemoveDuplicates could turn it all into a much more efficient one-liner.

Stylistically though, the code could use some help anyway.

The rather procedure begins with a good dozen variable declarations. Code is easier to follow when you don't need to constantly scroll up and down and back to know what's what: by declaring variables closer to their usage, you reduce the cognitive load and avoid that scrolling.

Dim xlApp As Application
Set xlApp = Excel.Application

Now, there's no need for this variable; Application is free, globally-scoped, and you're not owning that instance - just copying a reference to an existing object. Don't do that, use the globals that are right there waiting to be referenced instead. This includes any worksheet that exist at compile-time in ThisWorkbook:

Dim wsSource As Worksheet
Set wsSource = xlApp.ThisWorkbook.Worksheets("source")

Every worksheet has a CodeName property, that you can change in the properties toolwindow (F4). Locate that "source" worksheet in the Project Explorer, then look at its (Name) property - change it from what defaults to e.g. Sheet1, to a meaningful identifier, e.g. SourceSheet. And now you have a free, global-scope worksheet object referring to that specific sheet, readily usable everywhere in that VBA project - which makes this wsSource variable redundant as well.

Dim rBefore As Range
Set rBefore = wsSource.Range(wsSource.Cells(2, 1), wsSource.Cells(wsSource.UsedRange.Rows.Count, 1))

It's not clear whether that r stands for Range (in which case it's useless Hungarian Notation), and Before what this range is. So we try to look at other usages to infer its meaning... and we don't find any: the variable is assigned, but the assigned reference isn't used anywhere - the variable is redundant and can be removed entirely.

Dim ws As Worksheet
Set ws = xlApp.ThisWorkbook.Worksheets("testSheet")

Again, the CodeName could be TestSheet, and ws wouldn't be needed, provided that the worksheet exists at compile-time. Otherwise, kudos for fetching it from the Worksheets collection.

Dim r As Range
Set r = ws.Range(ws.Cells(2, 1), ws.Cells(ws.UsedRange.Rows.Count, 1))

UsedRange isn't a reliable way to get the last row of a column.

r isn't a meaningful name. testRange would tell us it's from the TestSheet, but wouldn't convey what it means - any single-letter variable name can never tell us anything whatsoever about something's purpose. Unless it's i for a For...Next loop counter, which any programmer knows about; here i isn't used like that, it's more like some processedRows.

DoEvents inside the loop, combined with updating the status bar at every single iteration, is contributing to slowing things down.

You're disabling ScreenUpdating, automatic Calculation, etc.; but you leave EnableEvents enabled, so a worksheet event is fired whenever you delete a row. You could use Union to combine all the rows you want to delete, and then delete them all in one single operation - instead of deleting them one at a time. But then again, Range.RemoveDuplicates makes that moot.

There are a number of issues (it's a constant work-in-progress), but Rubberduck could have picked up a number of the observations I made above. Kudos for Option Explicit, the meaningful name, and the explicit Public modifier on your macro!


Finding the minimum difference in a list of numbers I have written following code to find the minimum difference from a list of numbers.

Because I am using a loop once and LINQ again to find the minimum, the algorithm is O(N2).

Can you please tell me if I am using the framework in the most optimal way (speed and memory utilisation) to achieve this task:

using (StreamReader sr = new StreamReader("IN.in"))
using (StreamWriter sw = new StreamWriter("OUT.out"))
{
    int T = int.Parse(sr.ReadLine());
    for (int i = 1; i <= T; i++)
    {
        int N = int.Parse(sr.ReadLine());
        List<int> intList = sr.ReadLine().Split(' ').Select(e => int.Parse(e)).ToList();
        intList.Sort();

        List<int> diff = new List<int>();
        int leastDiff = int.MaxValue;
        for (int k = 0; k < intList.Count - 1; k++)
        {
            int iDiff = intList[k + 1] - intList[k];
            diff.Add(iDiff);
            leastDiff = Math.Min(leastDiff, iDiff);
        }
        sw.WriteLine(leastDiff);
    }
}

Benchmark

For 3 test case of 5 integers in each list where as for loop implementation takes 55±5 ms. Mr.Mindor LINQ implementation timing varies from 60±50 ms. Memory usage in both implementation is almost 8.3MB

You're worrying about the wrong problems: there isn't a lot more performance to be squeezed out of your code, but you could make substantial improvements to readability.

Choose a better file format
Why are you encoding the number of lines to be read into the file itself? That seems like redundant information. Just read all the lines there are and use those. You are also redundantly encoding the number of numbers per line: int N = int.Parse(sr.ReadLine());
You aren't doing anything with the parsed value, which is an indication that it shouldn't be in the file to start with.
Make your code reusable
Why not define the algorithm for finding the smallest difference inside an extension method? This allows you to separate the concerns of I/O and your minimum difference finding logic.

public static int SmallestDifference(this IEnumerable<int> source)
{
    var numbers = source as int[] ?? source.ToArray();
    Array.Sort(numbers);
    int difference = int.MaxValue;
    for (int i = 1; i < numbers.Length; i++)
    {
        difference = Math.Min(difference, numbers[i] - numbers[i - 1]);
    }
    return difference;
}

Be expressive
Using the deferred execution streaming File.ReadLines method together with an expressive LINQ query, you can write code that reads fluently and makes sense rather than confusing the reader with details:

var minDifferences = from line in File.ReadLines("IN.in")
                     let numbers = from number in line.Split(' ')
                                   select int.Parse(number)
                     select numbers.SmallestDifference().ToString();
File.WriteAllLines("OUT.out", minDifferences);

Now here's a tiny surprise left for the end:

  This solution consistently performs at least as well as your original code.
  (On average, it's a couple of milliseconds faster per fifty thousand lines, but that's in the realm of microbenchmarking which should be avoided. Just trying to give you a rough idea.)
  So stop worrying about performance; strive for clean code instead.

EDIT: I understand that you can't change the format (as per your comment), so I've added the code below to allow you to stick with your current file layout. The modification required is tiny: simply skip the first line, and only look for the smallest difference in lines that have more than one number.

var minDifferences = from line in File.ReadLines("IN.in").Skip(1)
                     let numbers = from number in line.Split(' ')
                                   select int.Parse(number)
                     where numbers.Count() > 1
                     select numbers.SmallestDifference().ToString();
File.WriteAllLines("OUT.out", minDifferences);


Money change/minimum coin algorithm How can I speed up this code?

coin=[50000, 20000, 10000, 5000, 2000, 1000, 500, 200, 100, 50, 20, 10, 5, 2, 1]
def matrix(coins, kerdes): # it create a metrix
    tabla = [[0 for m in range(kerdes + 1)] for m in range(len(coins) + 1)]

    for i in range(kerdes + 1):
        tabla[0][i] = i
    return tabla

def change_making(coin, change):
    table = matrix(coin, change)
    for c in range(1, len(coin) + 1):
        for r in range(1, change + 1):

            if coin[c - 1] == r:
                table[c][r] = 1

            elif coin[c - 1]> r:
                table[c][r] = table[c-1][r]

            else:
                table[c][r] = min(table[c - 1][r], 1 + table[c][r - coin[c - 1]])

    return table[-1][-1] #return the minimum coin what we need to make the change

newcoin=int(input()) # ask the new coin
coin.append(newcoin) #add to the coin list
coin.sort(reverse=True) #sort the list to get the biggest coin to first

changemakings=[int(x) for x in input().split()] # order the 'orders' to list for easier work with it
for changes in changemakings:

    print(change_making(coin, changes))

This is my code it calculate the minimum coin, with possible to give new coin which will result a non canonical change system.

I ran a few examples through the code and I have made the following assumptions:

the code calculates the minimum amount of coins needed to produce the user's input
the user can enter in a custom coin and it will use that as part of its calculations

Instead of using a matrix, I just found the highest coin which can be used to make change, then subtracted it from the total user inputted amount, and repeated it until there was no change left.

coins = [50000, 20000, 10000, 5000, 2000, 1000, 500, 200, 100, 50, 20, 10, 5, 2, 1]

coinToAdd = int(input("Enter in coin to add to coins:"))
userMoney = int(input("Enter in coin to calculate change:"))
coins.append(coinToAdd)
coins.sort(reverse=True)
totalCoins = 0
while [j for j in coins if userMoney >= j]:
  userMoney = userMoney - [j for j in coins if userMoney >= j][0]
  totalCoins = totalCoins + 1

print(totalCoins)


Summarize JS Array more efficiently I am working on a project for Udacity, where I should use CSV files to display data on fake corporate dashboard. Structure of the data I am working on is like this:

I am using AngularJS for this project, I created service for getting the data, formatting it to JSON and summarizing data into months. It works good, but I can see that it takes up to 1-2 sec to load a page, and data sheet has only around 90 lines. Anybody has suggestion how to speed it up?

I am also using MomentJS for formatting dates.

Service:

var getIssues = function() {
    return $http.get('data/issues.csv').then(
        function(response) {
            return CSV2JSON(response.data);
        }
    );
};

this.summorizeIssues = function() {

    return new Promise(function(resolve) {

        getIssues().then(function(data) {

            var datesarray = [];
            var returnarray = [];

            data.forEach(function(item) {
                if (item['submission timestamp']) {
                    var month = moment(item['submission timestamp'], 'M-D-YY h:m a').format('MMM-YY');
                    if (datesarray.indexOf(month) < 0) {
                        datesarray.push(month);
                    }
                }
            });

            datesarray.sort(function(a, b) {
                return moment(a, 'MMM-YY') - moment(b, 'MMM-YY');
            });

            datesarray.forEach(function(date) {
                var counter = 0;
                data.forEach(function(item) {
                    var month = moment(item['submission timestamp'], 'M-D-YY h:m a').format('MMM-YY');
                    if (month === date) {
                        counter += 1;
                    }
                });
                returnarray.push({
                    label: date,
                    issues: counter
                });

            });

            resolve(returnarray);
        });
    });
};

Controller:

issuesdataProvider.summorizeIssues().then(function(data) {
    _this.issues = data;
    _this.datesIssues = issuesdataProvider.monthstoYears(data);
    _this.currentDateIssues = _this.datesIssues[_this.datesIssues.length - 1];
});

This is the result of   return CSV2JSON(response.data); :

[
  {
    "submission timestamp": "4-7-16 2:28 PM",
    "customer name": "Tincom",
    "customer email address": "tincom@tincom.com",
    "description": "error 64",
    "open/closed status": "open",
    "closed timestamp": 0,
    "employee name": "Mickie Daley"
  },
  {
    "submission timestamp": "7-23-15 1:12 PM",
    "customer name": "Tripplehow",
    "customer email address": "tripplehow@tripplehow.com",
    "description": "error 95",
    "open/closed status": "closed",
    "closed timestamp": 42208.59706,
    "employee name": "Daine Whittington"
  },
  {
    "submission timestamp": "3-8-14 1:33 PM",
    "customer name": "Betadox",
    "customer email address": "betadox@betadox.com",
    "description": "error 94",
    "open/closed status": "closed",
    "closed timestamp": 41706.67644,
    "employee name": "Corliss Zhang"
  }]

And this how data looks after the process is done:

[
   {
      "label":"Jan-14",
      "issues":5
   },
   {
      "label":"Feb-14",
      "issues":1
   },
   {
      "label":"Mar-14",
      "issues":7
   },
   {
      "label":"Apr-14",
      "issues":4
   },
   {
      "label":"May-14",
      "issues":2
   },
   {
      "label":"Jun-14",
      "issues":2
   },
   {
      "label":"Jul-14",
      "issues":2
   },
   {
      "label":"Aug-14",
      "issues":1
   },
   {
      "label":"Sep-14",
      "issues":1
   },
   {
      "label":"Oct-14",
      "issues":1
   },
   {
      "label":"Nov-14",
      "issues":2
   }...
]

I am also using MomentJS for formatting dates.

I have experienced performance issues when using MomentJS before. It's not because it's terrible but rather the opposite. It just does a lot of things to get dates right. But you'll have to look out for potential bottlenecks.

  it takes up to 1-2 sec to load a page

Also consider that you're using AJAX. Network latency can become an issue as well. But there's nothing you can do with that since you have no control of the network. Consider other strategies like loading pages, or caching, or data compression.

We can do nothing with a slow network, but we can do something about the slow code.

        data.forEach(function(item) {
            if (item['submission timestamp']) {
                var month = moment(item['submission timestamp'], 'M-D-YY h:m a').format('MMM-YY');
                if (datesarray.indexOf(month) < 0) {
                    datesarray.push(month);
                }
            }
        });

        datesarray.sort(function(a, b) {
            return moment(a, 'MMM-YY') - moment(b, 'MMM-YY');
        });

        datesarray.forEach(function(date) {
            var counter = 0;
            data.forEach(function(item) {
                var month = moment(item['submission timestamp'], 'M-D-YY h:m a').format('MMM-YY');
                if (month === date) {
                    counter += 1;
                }
            });
            returnarray.push({
                label: date,
                issues: counter
            });

        });

The lag you are experiencing is most likely the browser's garbage collector kicking in. Every moment call creates a new MomentJS object. It's terrible thing to do in loops. In your case, you call it on each of the loops you have. You have 1 loop that executes n times, 1 sort whose callback is called an arbitrary amount of times, and 2-level loop which is n^2.

The goal here is to reduce the usage of moment and avoiding the use of nested loops. The following code is my attempt at it. It uses moment only once to grab everything it needs into a regular object. Everything else is just linear object/array operations.

const issuesByDate = data.reduce((data, item) => {
      // Return early when there's no timestamp
      if(!item['submission timestamp']) return data;

      // Group the data by using an object
      // { 'MMM-YY': { label: ..., timestamp: ..., issues: ... }}

      // The only time we use moment
      const itemMoment = moment(item['submission timestamp'])
      const label      = itemMoment.format('MMM-YY');
      const timestamp  = itemMoment.valueOf();

      // Use the label as key to group similar items
      // Append timestamp, a number value. Well need it later for sorting
      data[label] = data[label] || { label, timestamp, issues: 0 };
      data[label].issues += 1;

      return data;
    }, {});

             // Convert object into an array
             // [{ label: ..., timestamp: ..., issues: ... }]
return Object.keys(issuesByDate)
             .map(key => issuesByDate[key])
             // Sort here since object keys don't guarantee order
             .sort((a, b) => a.timestamp - b.timestamp)
             // Strip off the timestamp property that we don't need
             // [{ label: ..., issues: ... }]
             .map(item => ({ label: item.label, issues: item.issues }))

The code above can be further tuned by using regular loops instead of array methods, grouping up loop operations, use pushing to existing arrays instead of returning new arrays after each oepration. In summary, avoid calling out moment too often and avoid nested loops.