Vigenere cipher exercise in Haskell This is my implementation using the dreadful !!:

import Data.Char (chr, ord, toUpper)

-- A bit of self documentation help
type Key = String
type Msg = String

key :: Key
key = "TSTING"
msg :: Msg
msg = "I'm not even mad... This is impressive!"

-- | Checks if character is valid for encoding
isValid :: Char -> Bool
isValid c = let cUp = toUpper c :: Char
             in 'A' <= cUp && cUp <= 'Z'

-- | Given 'key' & 'msg' generate a list of [Maybe Int] indices
-- to map 'msg' from 'key', skipping invalid characters
toIdx :: Key -> Msg -> [Maybe Int]
toIdx k m = map (flip mod keyN <$>) $ toIdx_ 0 m
  where keyN = length k :: Int
        toIdx_ :: Int -> Msg -> [Maybe Int]
        toIdx_ _ "" = []
        toIdx_ acc (c:cs)
          | isValid c = Just acc : toIdx_ (acc + 1) cs
          | otherwise = Nothing : toIdx_ acc cs

-- | Given 'key' & 'msg' generate a list of numbers representing
-- the amount to shift 'msg' characters based on 'key'
toShifts :: Key -> Msg -> [Int]
toShifts k m = map toKey (toIdx k m)
  where kUp = map toUpper k :: Key
        toKey :: Maybe Int -> Int
        toKey Nothing  = 0
        toKey (Just x) = ord (kUp!!x) - ord 'A'

-- | Given 'by' & 'c', shift the Char 'c' by amount 'by'. 'by' can be both
-- positive & negative as well as 0.
shift :: Int -> Char -> Char
shift by c
  | isValid c && c >= 'a' = shift_ $ ord 'a'
  | isValid c && c >= 'A' = shift_ $ ord 'A'
  | otherwise = c
  where cONorm    = ord (toUpper c) - ord 'A' :: Int
        azN       = ord 'Z' - ord 'A' :: Int
        shift_ :: Int -> Char
        shift_ aO = chr $ aO + mod (by + cONorm) azN

-- Encode & decode a message using the given key.
vigenere, unVigenere :: Key -> Msg -> Msg
vigenere   k m = zipWith shift (toShifts k m) m
unVigenere k m = zipWith shift (map negate $ toShifts k m) m

I found that the most "annoying" thing when coming from background such as Python is to be able to keep track of things, for example when figuring out how to convert valid characters into usable positions to be then mapped with the key. That thing took me half a day to figure out!

How would you do it? Or is there some "standard" way of working with these sort of things? I'm referring particularly to toIdx & toIdx_, had to use toIdx_ to accumulate the indices with acc in this list of Maybe Int in order to correctly map valid Chars with key.

Of course, I wouldn't have it any other way but have an algorithm which takes any String input and creates an encoded String with preserved upper/lower-case and non-valid (out of ASCII alphabet) Chars.

To get rid of !! here, you can use it earlier and earlier until you never even generate an Int.

-- | Given 'key' & 'msg' generate a list of [Maybe Int] indices
-- to map 'msg' from 'key', skipping invalid characters
toIdx :: Key -> Msg -> [Maybe Char]
toIdx k m = toIdx_ (cycle $ map toUpper k) m
  where toIdx_ :: Key -> Msg -> [Maybe Char]
        toIdx_ _ "" = []
        toIdx_ key@(k:ey) (c:cs)
          | isValid c = Just k : toIdx_ ey cs
          | otherwise = Nothing : toIdx_ key cs

-- | Given 'key' & 'msg' generate a list of numbers representing
-- the amount to shift 'msg' characters based on 'key'
toShifts :: Key -> Msg -> [Int]
toShifts k m = map toKey (toIdx k m)
  where toKey :: Maybe Char -> Int
        toKey Nothing  = 0
        toKey (Just x) = ord x - ord 'A'

Of course, there is hardly a need to separate all these steps.

base :: Char -> Maybe Char
base c
  | 'a' <= c && c <= 'z' = Just 'a'
  | 'A' <= c && c <= 'Z' = Just 'A'
  | otherwise = Nothing

vigenere, unVigenere :: String -> String -> String
[vigenere, unVigenere] = (`map` [(+), (-)]) $ \direction k ->
  (.) snd $ (`mapAccumL` cycle k) $ \key@(k:ey) c -> case base c of
    Nothing -> (key, c)
    Just a -> (,) ey $ chr $ ord a +
      mod ((ord c - ord a) `direction` (ord (toUpper k) - ord 'A')) (ord 'Z' - ord 'A')


Listing page visits for each visitor I have some data in the following format:

Visitor_id    rownum    contenttype
abc             1          PageA
abc             2          PageB
abc             3          PageC
def             1          PageB
def             2          PageD

This data is for about 40 Million+ visitor_id. I want the result in the following format:

[['1PageA','2PageB','3PageC']
['1PageB','2PageD']]

I wrote the following code to achieve this:

import pandas as pd
import numpy as np
from tqdm import tqdm
d = d.sort_values('visitor_id')
flow_new = []
visitors = set(d['visitor_id'])
for visitor_id in tqdm(set(d['visitor_id'])):
    flow_session = []
    temp = d[d.visitor_id==visitor_id]
    for count in range(0,len(temp)):
        flow_session.append(str(temp.iloc[count,2]) + temp.iloc[count,3])
    flow_new.append(flow_session)

Although this code runs properly and gives me the required results on a smaller dataset, it takes insanely long on the whole dataset. Here's the expected time from tqdm:

  0%|            | 42/12009071 [01:47<7473:36:18,  2.24s/it]

This is obviously extremely long and there has to be another way to do this much quicker.

First thing first
Don't use tqdm just to show some fancy progress bar. On their website, they say the overhead is low (which it might be the case), but you can just print something like: Processing data. Please wait... until the data is processed.
Or, you can use timeit module to see how long your code takes.

Second, fix the bug:
This line:
flow_session.append(str(temp.iloc[count, 2]) + temp.iloc[count, 3])

Will throw an IndexError: single positional indexer is out-of-bounds because of temp.iloc[count, 3] which makes your code off-topic for this site's purpose.
Perhaps you wanted:
flow_session.append(str(temp.iloc[count, 1]) + temp.iloc[count, 2])

Instead of:
visitors = set(d['visitor_id'])

You can use:
visitors = d.visitor_id.unique()

In your for loop, you already have the unique visitors so why calculating them again ? Just do:
for visitor_id in visitors
    ...

Use len(df.index) instead of len(df). Due to one additional function call it is a bit slower than calling len(df.index) directly, but this should not play any role in most use cases.

You're not using numpy so you can remove it.

After I spent quite some time on this, I came up with a different approach which is probably better (and shorter):
Suppose you have the following dataframe:
d = pd.DataFrame(columns=['visitor_id', 'row_num', 'contenttype'], data=[
    ['abc', 1, 'PageA'], ['abc', 2, 'PageB'], ['abc', 3, 'PageC'],
    ['def', 1, 'PageB'], ['def', 2, 'PageD'], ['crf', 7, 'PageE']
])

You can create a new column which is the joint between rownum and content:
d["joint"] = d.row_num.astype(str) + d.contenttype

Then, you just want to group your data by visitor_id and create a list of lists with that and the joint. That can be easily done using a list comprehension and groupby:
result = [list(group.joint.values) for name, group in d.groupby("visitor_id")]

So, the final code looks like this:
import pandas as pd

def process_data(_df):
    _df["joint"] = _df.row_num.astype(str) + _df.contenttype

    return [list(group.joint.values)
            for name, group in d.groupby("visitor_id")]

if __name__ == '__main__':
    d = pd.DataFrame(columns=['visitor_id', 'row_num', 'contenttype'], data=[
        ['abc', 1, 'PageA'], ['abc', 2, 'PageB'], ['abc', 3, 'PageC'],
        ['def', 1, 'PageB'], ['def', 2, 'PageD'], ['crf', 7, 'PageE']
    ])
    print(process_data(d))

I don't think you can get any better than that.


Updating number of votes with a "Like" button I have a "Like" button in my iOS app that updates the votes every time it's tapped. Just like Facebook, Instagram, ProductHunt, etc.

I have a custom cell where that IBAction and IBOutlet is declared and that's where I'm making the query to the Parse db to update the UI (initially I'm querying the vote values from the other file from the cellForRowAtIndexPath.) In the custom cell file, I also created a delegate method, with a Protocol type that I'm accessing in my TableViewController in the cellForRowAtIndexPath.

I'm a bit of a newbie and studying the MVC data structure (assuming that's the biggest issue here), so please be easy on me!

Here is the code in the custom cell:

protocol CellButtonDelegate {

    func buttonClicked(cell : PFTableViewCell)

}
var delegate : CellButtonDelegate?

class FeedTableViewCell: PFTableViewCell {

    var delegate : CellButtonDelegate?

    var parseObject:PFObject?

    internal var buttonEnabled : Bool? {

        get {

            return loveOutlet.enabled

        } set {

            loveOutlet.enabled = newValue!
        }
    }

 @IBAction func loveButton(sender: AnyObject) {

        if(parseObject != nil) {

            if var votes: Int? = parseObject!.objectForKey("votes") as? Int {
                votes!++

                parseObject!.setObject(votes!, forKey: "votes")
                parseObject!.saveInBackground()

                print(votes!)
            }

            delegate?.buttonClicked(self)
        loveOutlet.enabled = false

        }

    }

Here is the code in the cellForRowAtIndexPath:

   override func tableView(tableView: UITableView, cellForRowAtIndexPath indexPath: NSIndexPath, object: PFObject?) -> PFTableViewCell? {

        var cell: FeedTableViewCell? = tableView.dequeueReusableCellWithIdentifier(cellIdentifier) as? FeedTableViewCell

        if(cell == nil) {

            cell = NSBundle.mainBundle().loadNibNamed("FeedTableViewCell", owner: self, options: nil)[0] as? FeedTableViewCell
        }

        objects![indexPath.row]

        if let pfObject = objectArray?[indexPath.row] {
                     cell?.parseObject = object
var votes: Int? = pfObject["votes"] as? Int
            if votes == nil {
                votes = 0
            }

            cell?.delegate = self

            let isEnabled = pfObject["isEnabled"] as! Int
            if isEnabled == 1 {

                cell?.buttonEnabled = true

            } else {

                cell?.buttonEnabled = false

            }

            print(pfObject["isEnabled"] as? Bool)
}

}

func buttonClicked(cell: PFTableViewCell) {

    let indexPath = tableView.indexPathForCell(cell)

    var object : PFObject = objectArray![indexPath!.row] as! PFObject

    object["isEnabled"] = 0
    objectArray?.removeAtIndex((indexPath?.row)!)
    objectArray?.insert(object, atIndex: (indexPath?.row)!)

}

Currently, when the button is tapped, the UI updates, but it also sends a query to the db (the activity indicator/internet indicator at the top right of the iPhone is spinning every time I "like" something).

When I use other apps, like Facebook for example, the query isn't being immediately sent and updated, it does it at a later point (don't understand when or how; can't figure that out but it does seem more efficient).

Lastly, sometimes an individual cell updates, and on other (awful) days the vote appears for every 3rd cell. Trying to figure this out for days.

In summary: ultimately, I want to optimize the query for voting in an iOS app. How do Facebook, Instagram and everyone else do it so smoothly?

Formatting
On the whole, there are several indentation problems through out your code.  Here is made the most obvious example:
        if let pfObject = objectArray?[indexPath.row] {
                     cell?.parseObject = object
var votes: Int? = pfObject["votes"] as? Int

The second line has two extra levels of indentation over what it should have.  The third line needs three more indentation levels to be in the correct place.
Xcode will pretty much put every single line of code exactly where it belongs.  All you have to do to get your indentation correct is simply not fight Xcode.  Even if you disagree with Xcode about how something should be indented, it's usually a lot easier to just do it Xcode's way because everyone else tends to give in to Xcode... and it's a lot less effort.  (For a personal example, I tend to feel like case statements within a switch should be indented one level from the switch statement, but Xcode disagrees... and it's a lot less work for me to just let Xcode have it its way.)

Delegates

var delegate : CellButtonDelegate?

Delegates should always be weak.  Without making a delegate weak, you're almost always setting up some sort of a retain cycle.  But what's more important, people expect for delegates to be marked as weak.  If you want to replicate the typical "delegate" pattern, but for some reason need the default strong reference, then you need to call your property by something other than "delegate".

Computed Properties
I have several comments about this property:
internal var buttonEnabled : Bool? {

    get {

        return loveOutlet.enabled

    } set {

        loveOutlet.enabled = newValue!
    }
}

First of all, internal is the default access level.  As you marked nothing else as public or private, everything in your code actually has this internal access level.  Above all else, we should prefer consistency.  If you're going to mark internal things with the internal keyword, you're now going to need to mark every property, method, extension, protocol, class, struct, enum, and function (and whatever else I forgot) with its access modifier.  Anything else is inconsistent.  But marking them all is too verbose.  We all know (or should know) that internal is the default level, and if we mark all the internal things as internal, it just becomes an unconstructive cluttered keyword.  I would only ever recommend marking something as internal in a class where basically everything else is already marked as public or private.  In that case, internal indicates "No, I didn't forget to mark this one--yes, it really is internal, I promise."
Next, there's a whole lot of white space in this.  In fact, you consistently have a ton of vertical white space throughout the entirety of your code.  Let's use white space more constructively, shall we?  Every single line of this property is all associated to this property.  By eliminating the blank lines in the middle, it helps the eye associate all of these lines together visually (in a way far more immediately obvious than the curly braces do).
Finally, let's talk about optionals... Your loveOutlet doesn't appear to be defined in this file, so I don't know if it is an optional or if it is a non-optional but the enabled property is optional.  I don't know what has led you to marking this buttonEnabled property as a Bool? rather than a non-optional Bool, but based on the force-unwrapping in your setter, it seems clear that this should probably be a non-optional.  We should allow the user to pass nil for the buttonEnabled property--how can that even make sense?  It clearly doesn't make sense, because if they ever do, we simply crash.  This is supremely bad design.  Swift gives you the power to clearly dictate which properties can and cannot be set to nil.  What you've done here is created a property that can be set to true, false, or "crash my app please".
So more realistically, this property should look about like this:
var buttonEnabled: Bool {
    get {
        return loveOutlet.enabled
    }
    set {
        loveOutlet.enabled = newValue
    }
}

If loveOutlet is optional (in this case, for it to compile, it'd have to be an implicitly unwrapped optional), or its enabled property is optional, then we can go for one of these two options in the get:
loveOutlet?.enabled ?? false

or
loveOutlet.enabled ?? false

if true then true else false
let isEnabled = pfObject["isEnabled"] as! Int
if isEnabled == 1 {

    cell?.buttonEnabled = true

} else {

    cell?.buttonEnabled = false

}

Here, we see another issue with overuse of vertical white space, poor optional management, and a pattern that I simply don't understand: if true then true else false.
This as! Int can cause an app crash in multiple ways.  First, the value can simply not exist for the key.  Second, the value could exist, but not be convertible to an Int.  If this is a dictionary representing JSON, the value could be a string, floating point, or bool.  If it's not representing JSON, it could be anything.  But we can rewrite this safer and using significantly fewer lines:
cell?.buttonEnabled = (pfObject["isEnabled"] as? Int) == 1

Although, arguably, we'd really want to fix our model so that the value for the "isEnabled" key is actually a bool, then we can write something that makes a bit more sense:
cell?.buttonEnabled = (pfObject["isEnabled"] as? Bool) ?? false

Optional Safety
It's really unfortunate that being safe with optionals is an optional thing at all.  It seems mostly a hold-over from the short period of time after Swift was introduced and before Objective-C's nullability annotations.
Despite this, we should still strive to always be as safe as possible with our optionals.  Force-unwrapping nil is always a crash.
if(parseObject != nil) {

        if var votes: Int? = parseObject!.objectForKey("votes") as? Int {
            votes!++

            parseObject!.setObject(votes!, forKey: "votes")
            parseObject!.saveInBackground()

            print(votes!)
        }

        delegate?.buttonClicked(self)
    loveOutlet.enabled = false

    }

Here, we have not been safe with our optionals.  And the worst part here is that there's not even an imaginably good reason for this lack of safety--we're checking for nil in an if statement and only force-unwrapping within that block.
The most dangerous problem with this is that it's not thread safe.  If thread 1 checks that foo is non-nil, finds it is not, then enters the if block, then thread 2 sets foo (an instance variable) to nil, then back on thread 1, we're in the if block and force unwrap, we've now crashed (despite having checked against nil).  This is why we should prefer the if let construct.
if let pObj = parseObject {
    // safely use pObj as a constant, unwrapped, thread-safe, locally-scoped version of parseObject
}

The next line is particularly strange.  It's a clear indication that we're aware of the if let/if var syntax, but we've used it in a mysterious and strange way.
(For the record, I absolutely never want to see votes!++ in my code base, and if a developer on my project checked that code in, I'd be looking to move that dev to a different platform.  It's horrendous...)
You've used if var to properly check for nil... but then you specify the type as an optional which still has to be unwrapped, despite having already checked against nil.  If we simply don't specify the type, votes would be a non-optional that doesn't need to be force unwrapped.
We should also generally prefer let variables to var variables.  In fact, we should default to let and only change it to var when we absolutely must modify the value.  I don't think votes is something we absolutely must modify.
Finally, keep in mind we can unwrap multiple values in a single if let statement.  I'd prefer to see this code snippet as something more like this:
if let
    pObj = parseObject,
    votes = pObj.objectForKey("votes") as? Int {
        pObj.setObject(votes + 1, forKey: "votes")
        pObj.saveInBackground()
}
delegate?.buttonClicked(self)
loveOutlet.enabled = false

(The actual logic for when to call buttonClicked() and set the outlet disabled may be slightly different, but I wanted to provide you with this example.)


Solving the Lazy Hobo Riddle I would like help in optimizing this calculation, which is from the Lazy Hobo Riddle:

  There once were 4 hoboes travelling across the country.  During their
  journey, they ran short on funds, so they stopped at a farm to look
  for some work.  The farmer said there were 200 hours of work that
  could be done over the next several weeks.  The farmer went on to say
  that how they divided up the work was up to them.  The hoboes agreed
  to start the next day.

  The following morning, one of the hoboes - who was markedly smarter
  and lazier than the other 3 - said there was no reason for them all to
  do the same amount of work.  This hobo went on to suggest the
  following scheme:

  The hoboes would all draw straws.
  A straw would be marked with a number.
  The number would indicate both the number of days the drawer must work and the number of hours to be worked on each of those days.  For
  example, if the straw was marked with a 3, the hobo who drew it would
  work 3 hours a day for 3 days.

  It goes without saying that the lazy hobo convinced the others to
  agree to this scheme and that through sleight of hand, the lazy hobo
  drew the best straw.

  The riddle is to determine the possible ways to divide up the work
  according to the preceding scheme.

It basically just involves finding all possible solutions of \$a,b,c,d\$ such that \$a^{2} + b^{2} + c^{2} + d^{2} = 200 \$

for(int a = 1;a<=100; a++)
    for(int b = 1; b<=100; b++)
        for(int c = 1; c<=100; c++)
            for(int d = 1; d<=100; d++){
                ++counter;
                if(a*a + b*b + c*c + d*d == 200)
                    cout << "a: " << a << " b: " << b << " c: " << c << " d: " << d << endl;
            }

Notice how the square of a number 15 or greater exceeds 200? What you can do, is set the interval from 1 to 14.
There is no advantage in evaluating the same combination over and over again. Realize that the most efficient way is to structure your for loops such that

$$ a  \leq b  \leq c  \leq d $$

In your attempt, you are iterating 38,416 times!

By limiting the interval, you iterate just 2380 times!

One more thing you can do: check and break when \$a^{2} + b^{2} + c^{2} + d^{2} > 200 \$

for(int a = 1; a<=14; a++)
    for(int b = a; b<=14; b++)
        for(int c = b; c<=14; c++)
            for(int d = c; d<=14; d++){
                if(a*a + b*b + c*c + d*d == 200)
                    cout << "a: " << a << " b: " << b << " c: " << c << " d: " << d << endl;
                else if(a*a + b*b + c*c + d*d > 200)
                    break;
            }

Now you iterate only 1,214 times! This is way more efficient.


Sorting an unsorted array This is a very simple sorting logic that I have written and it's working also.
But Is there anything wrong in the traversing of the array?
Or the code is not optimized?
The code is simply declaring an array of size N.
And then sorting is performed by traversing.

import java.util.Scanner;
import java.io.*;
class BubbleSort
{
    public static void main(String args[])throws IOException
    {
        int size,temp=0;
        System.out.println("Enter the size of the array");
        Scanner sc=new Scanner(System.in);
        size=sc.nextInt();
        int arr[]=new int[size];
        System.out.println("Enter the elements in the array");
        for(int i=0;i<size;i++)
        {
            arr[i]=sc.nextInt();
        }
        for(int i=0;i<size;i++)     //Here is the sorting logic
        {
            for(int j=i+1;j<size;j++)
            {
                if(arr[i]>arr[j])
                {
                    temp=arr[i];
                    arr[i]=arr[j];
                    arr[j]=temp;

                }
            }
        }
        for(int i=0;i<size;i++)
        {
            System.out.print(arr[i]);
        }
     }
    }

While this code works as intended, there is room for improvement.

Use small functions that do one thing

The main method does too many things:

Read input from the console
Sort the array
Print the array

These steps are distinct features, their implementations are independent from each other.
It would be better if they were in separate functions.
That will make the responsibilities of each function clear,
and easy to test and verify the correct behavior.

Declare and initialize variables right before you need them

Instead of this:

int size, temp = 0;
System.out.println("Enter the size of the array");
Scanner sc = new Scanner(System.in);
size = sc.nextInt();

You could declare and initialize size where needed:

System.out.println("Enter the size of the array");
Scanner sc = new Scanner(System.in);
int size = sc.nextInt();

Notice that I completely dropped temp.
It's not needed here.
You can declare it later when needed.

Unnecessary throws

The main method declares to throw IOException but it's unnecessary.

Style

Although this is a valid way to declare an array in Java, it's unusual:

int arr[] = new int[size];

This is the recommended writing style:

int[] arr = new int[size];

The reasoning is that [] is part of the type.
In int arr[] the type and the variable name are mixed,
in int[] arr the type is cleanly at the left,
and the variable name is cleanly at the right.

The same goes for String args[] in the arguments of main.


Finding words of a given length only using given letters It's a (simple) guesser for game where you need to find words of a given length only using given letters. I'd like to know if there's a more pythonic way to do things here.

#!/usr/bin/env python3
import sys

if len(sys.argv) < 3:
    sys.exit("Usage: " + sys.argv[0] + " letters number_of_letters [dictionnaryfile]")

letters = sys.argv[1].upper()
wordsize = int(sys.argv[2])
dictionnary_name =  "dict.txt" if len(sys.argv) < 4 else sys.argv[3]

try:
    wordlist = open(dictionnary_name).read().split('\n')
except FileNotFoundError:
    sys.exit("Couldn't find dictionnary file \"" + dictionnary_name + "\"")

good = []
for w in wordlist:
    if len(w) == wordsize:
        up = w.upper()
        ok = True
        for c in up:
            if not c in letters:
                ok = False
                break
        if ok:
            good.append(w)

print("Found " + str(len(good)) + " results:")
print(good)

The following code shows I would have done what you asked in Python 2.7. Other readers will tell you the ways in which it is a) not Pythonic and b) wrong.

Start with a module comment describing what the code is intended to do. (This is available to the code as __doc__.)
Use the various Python string delimiters to avoid escaping.
Name variables consistently.
strip() text strings read from a text file.
Make letters a set to reflect its usage.
Create the list of matching words with a list comprehension.

Here is the code.

"""
    Find all the words in a reference dictionary with a specified number of letters
    that contain all the letters in a specified list of letters.

    The number of letters and the list of letters must be specified on the command line.

    The reference dictionary may be specified on the command line. If is not then
    dict.txt is used.

    The matching of letters is case-insensitive.
"""
import sys

if len(sys.argv) < 3:
    print >> sys.stderr, __doc__
    sys.exit('Usage: %s letters number_of_letters [dictionary_file]' % sys.argv[0])

letters = set(sys.argv[1].upper())
wordsize = int(sys.argv[2])
dictname = 'dict.txt' if len(sys.argv) <= 3 else sys.argv[3]

try:
    wordlist = [w.strip() for w in open(dictname).readlines()]
except IOError:
    sys.exit('''Couldn't find dictionary file "%s"''' % dictname)

matches = [w for w in wordlist
             if len(w) == wordsize
                and all(c in letters for c in w.upper())]

print 'Found %d results:' % len(matches)
print matches


An enum that has situationally invalid entries Is it considered bad style to have that BOTH value in the Direction enum? Problems can arise from BOTH being present, for example when trying to build a procedural UI for the Car class, it (the generator) would see the setTurnSignal(Direction) method and create a way for the user to give it a Direction, but it would suggest all values, of which one is illegal.

Conversely, not having BOTH causes issue with the getBlinkerStatus() as it is no longer able to indicate that both are on. There are a couple ways I could address that issue, such as creating more blinker getters, but that  or switching form an enum to bit based values and building masks for them.

enum Direction{
    LEFT, RIGHT, BOTH, NONE;
}

class Car{
    Direction turnSignal;
    boolean hazards;

    void setTurnSignal(Direction direction){
        if(Direction.BOTH.equals(direction)){
            throw new IllegalArgumentException("Turn signal cannot be set to the given direction: " + direction);
        }else{
            turnSignal = direction;
        }
    }

    Direction getBlinkerStatus(){
        if(hazards){
            return Direction.BOTH;
        }else{
            return turnSignal;
        }
    }

There are a couple ways I could address that issue, such as creating more blinker getters, but that or switching form an enum to bit based values and building masks for them.

java.util.EnumSet<E> is the piece you are looking for here.

public class Car {
    static enum Direction {
        LEFT, RIGHT
    }

    EnumSet<Direction> turnSignal;

    void setTurnSignal(Direction direction) {
        turnSignal = EnumSet.of(direction);
    }

    void setHazards() {
        turnSignal = EnumSet.of(Direction.LEFT,Direction.RIGHT);
    }

    EnumSet<Direction> getBlinkerStatus() {
        return turnSignal;
    }
}

That said, this probably isn't the right approach.  If you are trying to capture the state of the blinkers, then you should probably be implementing a state machine that tracks what is going on.  A state machine is absolutely what you want if you are worried about the hazards being enabled and disabled when the turn signal is going (as you indicate in your comment).

The straight forward model would be that you have six states - [LEFT,NONE,RIGHT] x [HAZARDS ON, HAZARDS OFF] - with simple transitions between them.

You can hand roll a state machine easily enough, but beware -- it's common for state machines to snowball in complexity, as you learn more about the system you are modeling.  So it might be worth investing in a library like stateless4J, which gives you a DSL for wiring together your state transition diagram.

public class FSM {
    enum State {
        OFF, LEFT, RIGHT, HAZARDS
        // Unimaginitive names for the "hidden" states.
        , HAZARDS_LEFT, HAZARDS_RIGHT
    }

    enum Toggle {
        LEFT, RIGHT, HAZARDS
    }

    static StateMachine<State, Toggle> create() {
        StateMachine<State, Toggle> fsm = new StateMachine<State, Toggle>(State.OFF);
        fsm.configure(State.OFF)
                .permit(Toggle.LEFT, State.LEFT)
                .permit(Toggle.RIGHT, State.RIGHT)
                .permit(Toggle.HAZARDS, State.HAZARDS);
        fsm.configure(State.HAZARDS)
                .permit(Toggle.LEFT, State.HAZARDS_LEFT)
                .permit(Toggle.RIGHT, State.HAZARDS_RIGHT)
                .permit(Toggle.HAZARDS, State.OFF);
        // etc.

        return fsm;
    }
}

Alternatively, you might consider a state machine design where some of these states are substates of others; stateless4j has substate support, but I haven't explored it enough to have an opinion on whether this should apply for your problem.


Refactoring of this complex PHP method I have one complex method with 1890 paths.  I don't know how can be this refactored. Can somebody add some tips about it?  Maybe the MySQL table structure is wrong.

public function fetchClubChallengeState()
{
    $c = $this->caller->in_club;
    $o = $this->opponent->in_club;

    if (!$c || !$o) {
        return false;
    }

    $last = Yii::app()->db->createCommand()
        ->select('*')
        ->from('challenge')
        ->where('(caller=:caller OR caller=:opponent) AND winner=0', [':caller'=>$c, ':opponent'=>$o])
        ->limit(1)
        ->queryRow();

    if ($last['caller']==$c and $last['opponent']==$o or $last['caller']==$o and $last['opponent']==$c) {
        $created = strtotime($last['created']);

        if ($this->isBetweenDates($created + 1800, $created + 3600)) {
            $this->challengeID = (int)$last['id'];
            $this->callersClubRole = $last['caller'] == $c ? 'caller' : 'opponent';
            $this->callersClub = $last['caller'] == $c ? $last['name_caller'] : $last['name_opponent'];
            $this->opponentsClubRole = $last['opponent'] == $o ? 'opponent' : 'caller';
            $this->opponentsClub = $last['opponent'] == $o ? $last['name_opponent'] : $last['name_caller'];
        }
    }
}

Edit:
The structure of the challenge table:

CREATE TABLE `challenge` (
 `id` int(11) NOT NULL AUTO_INCREMENT,
 `caller` mediumint(7) unsigned NOT NULL DEFAULT '0',
 `opponent` mediumint(7) unsigned NOT NULL DEFAULT '0',
 `name_caller` varchar(20) NOT NULL DEFAULT '',
 `name_opponent` varchar(20) NOT NULL DEFAULT '',
 `winner` tinyint(1) unsigned NOT NULL DEFAULT '0',
 `created` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
 //...
PRIMARY KEY (`id`),
KEY `c_o` (`caller`,`opponent`),
KEY `winner` (`winner`)
) ENGINE=InnoDB AUTO_INCREMENT=11 DEFAULT CHARSET=utf8;

If the challenge was started by Player1's club, that clubs ID goes to the caller field and the ID of the opponents club goes to the opponent field. These two IDs are checked here:
if ($last['caller']==$c and $last['opponent']==$o or $last['caller']==$o and $last['opponent']==$c) {
This means, that the found challenge is between the clubs of the 2 players.

The code is also here.

Don't know what happens in your db, but LIMIT 1 suggests that result of your query will return exactly what you ask or nothing. In this case no need to compare caller and opponent in if statement - just return (false) from method when $last is empty/false.

Otherwise if one caller may have more 'winner = 0' opponents LIMIT 1 could give you result that you won't use (if statement will reject it) while there's a valid one but was cut off by limit. In this case better provide full description in where clause (if statement will also check if there is any result).


RPi script to periodically collect data from Arduino through serial port I'm not used to Python.

The script opens serial communications with an Arduino unit and send commands to and receives information from the Arduino. It is intended to run indefinitely until the system is shut down or pulled out of field deployment/laboratory testing.

The time.sleep() calls with offsets are assurances that repeated calls won't eventually result to time drifts, as mentioned in this Stack Overflow Q/A. I don't trust myself with this. I need confirmation that I placed the start_time = time.time() lines in proper locations. This time drift issue is crucial because the script shall run for months.

read_data() sends a command to the Arduino, which tells it to give the Pi new data. Data sets written by the Arduino vary in length, so I can't use serial.read(expected_bytecount). Data sent between start and stop bytes (e.g. *data1,data2,data3#) is always valid, so I don't need to do checks there. Is my algorithm here enough for my task?

wait_until_sampling() keeps the Pi from being busy around the clock. Reading data occurs every 15th and 45th minutes of an hour. I thought of using minutes % 15 != 0, but there are the 0th and 30th marks. I also tried doing away with datetime but I can't get the minutes of the system time from time. Is there a simple way to improve this algorithm by dropping datetime?

import datetime
import time
import serial

def read_data():
    data = []
    start_time = time.time()
    arduino.write(b'S')
    time.sleep(360.0 - ((time.time() - start_time) % 360.0))
    while True:
        temp = arduino.read().decode(encoding='UTF-8')
        if temp  == "*":
            while True:
                temp = arduino.read().decode(encoding='UTF-8')
                if temp == "#":
                    break
                else:
                    data.append(temp)
            break
    return data

def wait_until_sampling():
    start_time = time.time()
    dt_now = datetime.datetime.now().time()
    while (dt_now.minute != 45) and (dt_now.minute != 15):
            time_now = time.time()
            dt_now = datetime.datetime.now().time()
            time.sleep(1.0 - ((time_now - start_time) % 1.0))

arduino = serial.Serial('/dev/ttyACM0', 9600)
time.sleep(20.0)
data_returned = []
while True:
        wait_until_sampling()
        data_returned = read_data()
        # post-process data_returned here

A simple technique to avoid busy wait is to

import select
....
select.select([], [], [], timeout)

The timeout is simply target_time - current_time in seconds. Since select interprets it as a floating point, the granularity is in  milliseconds at least.

I see no possibility for time drift.
The nested loops in read_data do not look right. Consider instead

def read_data():
    data = []

    temp = None
    while temp != '*':
        temp = arduino.read()...

    while temp != '#':
        temp = arduino.read()...
        data.append(temp)

    return data[:-1]

I am not sure why do you sleep() before reading the response.


DFS in a binary tree Problem statement: Perform a depth first traversal on a BST and print the elements traversed.

class Node(object):
    def __init__(self, value, left=None, right=None):
        self.value = value
        self.left = left
        self.right = right

class BST(object):
    def __init__(self, value):
        self.root = Node(value)

    def insert(self, value):
        current = self.root
        while current:
            if value > current.value:
                if current.right is None:
                    current.right  = Node(value)
                    break
                else:
                    current = current.right
            else:
                if current.left is None :
                    current.left  = Node(value)
                    break
                else:
                    current = current.left

    def preorder(self,root):
## root,left,right

        if root:
            print root.value
            if root.left:
                self.preorder(root.left)
            if root.right:
                self.preorder(root.right)

    def postorder(self,root):
## left, right, root

        if root:
            if root.left:
                self.postorder(root.left)
            if root.right:
                self.postorder(root.right)
            print root.value

    def inorder(self,root):
## left, root, right

        if root.left:
            self.inorder(root.left)
        print root.value
        if root.right:
            self.inorder(root.right)

t = BST(100)
t.insert(12)
t.insert(92)
t.insert(112)
t.insert(123)
t.insert(2)
t.insert(11)
t.insert(52)
t.insert(3)
t.insert(66)
t.insert(10)
print "preorder traversal is this "
t.preorder(t.root)
print "postorder traversal is this "
t.postorder(t.root)
print "inorder traversal is this "
t.inorder(t.root)

How can this code be more efficient?

This code is pretty much the most efficient it can be, so I will add some pointers on the coding style and convention.

Instead of calling your Binary Search Tree class a BST, define it as

class BinarySearchTree(object):
    # remainder of your code

Gives more meaning to the class name.
The preorder traversal is called like t.preorder(t.root). But wait, t already has the root node in it, so it would be much more readable if we could do t.preorder(). So modify the signature to:

def preorder(self, root=None):
    if root is None:
        root = self.root
    # remainder of your code

Same should be done for t.inorder() and t.postorder() as well.

Spaces: You need a space between your arguments, as recommended by PEP-0008. So do def inorder(self, root): instead of def inorder(self,root):. This goes for the inorder and postorder as well
Using root as the argument name is misleading - its better to call it as node, which has the default value of self.root. This is cleaner, since in the DFS, you would traverse all the nodes of the tree.

def preorder(self, node=None):
    # replace root with node in your code

Same should be done for inorder and postorder as well.
Comments and indentation: Your comments are currently not indented right. Instead of having

    def inorder(self,root):
## left, root, right

Try defing them in the form of docstrings

def inorder(self,root):
    """traversal order: left child node, current node, right child node"""
    # remainder of your code


Login System In Batch Here I made a program in batch that detects all files with the .user extension. Then it allows the user to pick a username by entering the number associated with that username.

The code is messy, so I will explain.

@echo off
setlocal ENABLEEXTENSIONS ENABLEDELAYEDEXPANSION
:login_menu
cls
set x=1
set users=
cd "users"
for %%A in (*.user) do (
echo !x!. %%~nA
set users=%%~nA:!x!,!users!
set /a x=!x!+1
)
echo.
set /p ch=Select User:
if "%ch%" == "" (goto login_menu)

for %%B in (%users%) do (
    for /f "tokens=1,2 delims=:" %%C in ("%%B") do (
    set userNumber=%%D
    set userN=%%C
    if !ch! == %%D goto password
    echo BDEV: %%B
    pause
)
)
echo.
echo That user doesn't exist!
pause
goto login_menu

:password
cls
cd "users"
for /f "tokens=1,2 delims=:" %%E in (!userN!.user) do (set password=%%F)
echo Enter your password, !userN!
echo.
set /p password1=Password:

if %password1% == %password% goto menu
echo.
echo That password is invalid!
pause
goto password

:menu
echo Hey! You're logged in as !userN!
pause

The variable x is going to be the number which the username will be associated with. The variable users Makes sort of a 'map' to usernames to the number associated with that username

In the first for loop, it gets all the files in the folder users. It echo's out all the usernames that the user can pick.

The second for loop goes into the variable users and separates the usernames with their respective numbers. For example: if I have the usernames admin:4,steve:3,john:2,jane:1 it will separate them into admin:4 and steve:3 and john:2 and jane:1.

The third for loop (which is in the second for loop) separates each username to number into separate variables. For example, if we have the username Collins with the number 3, it will put the username Collins in the userN variable and the number 3 into the variable userNumber.

It then checks what number the user selected.

The final for loop goes into the user file that the user has selected. So if the user selected admin it will go to the user file admin.user and find the password.

If the password is invalid, it rejects access. If it correct it allows access.

Is there any way to make it less like spaghetti code?

If you need more explanation, I will be happy to provide more information.

My main recommendation is that your variables have more descriptive names. While !x! may seem like a reasonable variable name for a counter that you're never going to use again, calling it something like !counter! makes your code easier for other people to maintain when there's a lot more of it.

I also got rid of ENABLEEXTENSIONS because that's enabled by default, and I moved cd users above :login_menu because if somebody enters an invalid user number and you're already in the users folder, your code will try to go into the users folder that's inside of the users folder and since there isn't one, you'll get an error.

To cut out that nested for loop, I stored the username map in an array. From there, you can determine whether or not a username is valid by if the variable exists.

Finally, I replaced the !s that you echoed with ^^!s so that they would be escaped. Because you have delayed expansion enabled, the last bit of your code would be displayed as HeyuserN because batch would consider ! You're logged in as ! to be a variable (because that's a valid variable name in batch).

If you could guarantee that you would never have more than ten users, you could build a string for a choice list to guarantee that the user would never enter an invalid user number, but I assumed this would be used by a large number of people.

@echo off
setlocal enabledelayedexpansion

cd users

:login_menu

set "user_counter=1"
for %%A in (*.user) do (
    echo !user_counter!. %%~nA
    set "users[!user_counter!]=%%~nA"
    set /a user_counter+=1
)
echo.

set /p "user_selection=Select user: "
if "%user_selection%"=="" goto login_menu
if not defined users[%user_selection%] (
    echo That user does not exist^^!
    pause
    goto login_menu
)
set "user_name=!users[%user_selection%]!"

:enter_password
for /f "tokens=1,2 delims=:" %%E in (!user_name!.user) do set "stored_password=%%F"
echo Enter your password, !user_name!
echo.
set /p "entered_password=Password: "

if "%entered_password%"=="%stored_password%" goto menu
echo.
echo That password is invalid^^!
pause
goto enter_password

:menu
echo Hey^^! You're logged in as !user_name!^^!
pause


Find the intersection of ArrayList elements inside an ArrayList So the structure is {{Dog,Cat,Human},{Human,Dog,Whale,rabbit,Cow},{Monkey,Human,Dog}}.

Output should be: Dog,Human.

I have to find the intersection of ArrayList elements (Strings, for now. Could be integers as well) inside the bigger ArrayList. Previously, I have seen codes of finding the intersection of separate ArrayLists, but not sure how I could do it inside the same ArrayList.

For separate ArrayLists a code like the following works. But how do I make it work for multiple ArrayLists inside one ArrayList? I was asked this in an interview. Worked for separate lists, but when the interviewer asked this, I couldn't code it out for the same ArrayList.

public class Test {

    public <String> List<String> intersection(List<String> list1, List<String> list2) {
        List<String> list = new ArrayList<String>();

        for (String t: list1) {
            if(list2.contains(t)) {
                list.add(t);
            }
        }

        return list;
    }
public static void main(String[] args) throws Exception {

        List<String> list1 = new ArrayList<String>(Arrays.asList("Dog", "Cat", "Human"));
        List<String> list2 = new ArrayList<String>(Arrays.asList("Human", "Dog", "Whale", "rabbit", "Cow"));

        System.out.println(new Test().intersection(list1, list2));

    }
}

This produces correct output for two separate ArrayLists.

Approaching programming interview questions

One of the most important things during programming interviews is clarifying the problem.
If the task is intersection of lists as opposed to sets,
an important question is how to handle duplicate values.

For example, the current implementation is buggy.
The intersection method will return a,a,a for the input a,a,a and a,a, but it will give a,a for the input a,a and a,a,a.
The logical assumption would be that it should always return a,a regardless of the order of parameters.
Then again, you should not just assume things,
you should ask for the expected behavior,
or at least state your assumptions loud and clear.

And if there can be no duplicates in the inputs,
then why aren't they sets instead of lists?

Why is ArrayList specified instead of just List?
ArrayList is a specific implementation,
when List is the general concept.

Generic type tokens

String, the name of a common class in the JDK,
and written in PascalCase,
is a terrible choice for a generic type token in this code:

public <String> List<String> intersection(List<String> list1, List<String> list2) {
    List<String> list = new ArrayList<String>();

    for (String t: list1) {
        if(list2.contains(t)) {
            list.add(t);
        }
    }

    return list;
}

It should have been written like this:

public static <T> List<T> intersection(List<T> list1, List<T> list2) {
    List<T> list = new ArrayList<>();

    for (T t : list1) {
        if (list2.contains(t)) {
            list.add(t);
        }
    }

    return list;
}

Intersection of many lists

  But how do I make it work for multiple ArrayLists inside one ArrayList?

I'm not sure I get what's blocking you.
Perhaps you mean, instead of the intersection of 2 lists,
how to intersect more than 2?

In that case,
that's an interesting question to discuss about,
and consider different alternatives,
starting from simple (brute force) techniques,
and then trying to improve.

Simple solution: if you have n lists, you could intersect the first 2,
then intersect the result with the 3rd,
then intersect the result with the 4th,
and so on, until the last.
The final result will be the intersection of all the lists,
reachable in n - 1 steps.

Is it possible to do better?

Hint 1: consider the concept of divide and conquer.
Hint 2: how does merge sort work?


Preprocessing text input to a machine-learning algorithm I have written the following function to preprocess some text data as input to machine learning algorithm. It lowercase, tokenises, removes stop words and lemmatizes, returning a string of space-separated tokens. However, this code runs extremely slowly. What can I do to optimise it?

import os
import re
import csv
import time
import nltk
import string
import pickle
import numpy as np
import pandas as pd
import pyparsing as pp
import matplotlib.pyplot as plt
from sklearn import preprocessing
from nltk.corpus import stopwords
from nltk.stem import PorterStemmer
from nltk.stem import WordNetLemmatizer
from nltk.tokenize import sent_tokenize, word_tokenize
from sklearn.feature_extraction.text import CountVectorizer
from sklearn.feature_extraction.text import TfidfTransformer

def preprocessText(text, lemmatizer, lemma, ps):
        '''
        Lowercase, tokenises, removes stop words and lemmatize's using word net. Returns a string of space seperated tokens.
        '''
        words = text.lower()
        words = re.sub("[^a-zA-Z]", " ", words)
        words = word_tokenize(words)
        stemmed_words = []
        stops = set(stopwords.words("english"))
        meaningful_words = [w for w in words if not w in stops]
        text = ""
        if lemmatizer == True:
            pos_translate = {'J':'a', 'V':'v', 'N':'n', 'R':'r'}
            meaningful_words = [lemma.lemmatize(w,pos=pos_translate[pos[0]] if pos[0] in pos_translate else 'n') for w,pos in nltk.pos_tag(meaningful_words)]
            for each in meaningful_words:
                if len(each) > 1:
                    text = text +" " + each
            return text
        else:
            words_again = []
            for each in meaningful_words:
                words_again.append(ps.stem(each))
            text = ""
            for each in words_again:
                if len(each) > 1:
                    text = text +" " +each
            return(text)

Given that you are already using Python, I would highly recommend using Spacy (base text parsing & tagging) and Textacy (higher level text processing built on top of Spacy).  It can do everything you want to do, and more, with one function call:

http://textacy.readthedocs.io/en/latest/api_reference.html#textacy.preprocess.preprocess_text

For your further travels in text based machine learning, there are also a wealth of additional features, particularly with Spacy 2.0 and its universe.


Using LINQ to group social networking links for each user I have a model named User. Each user can have some social links. Now I want to get all users with their social link as a single row in the result. Here are my models:

public class User
{
    public int Id { get; set; }
    public string Name { get; set; }
    public ICollection<SocialLink> SocialLinks { get; set; }
}

public class SocialLink
{
    public string Url { get; set; }
    public SocialType Type { get; set; }
    public User User { get; set; }
    public int UserId { get; set; }
}

public enum SocialType
{
    FaceBook    = 0,
    Twitter     = 1,
    Youtube     = 2,
    Linkedin    = 3,
    GooglePlus  = 4,
    Telegram    = 5,
    Instagram   = 6
}

public class UserViewModel
{
    public string Name { get; set; }
    public string Facebook { get; set; }
    public string Twitter { get; set; }
    public string Youtube { get; set; }
    public string Linkedin { get; set; }
    public string GooglePlus { get; set; }
    public string Telegram { get; set; }
    public string Instagram { get; set; }
}

I want my query to be like this in the result:

Name    | Facebook | Twitter  | Youtube | Linkedin | GooglePlus | Telegram | Instagram
======================================================================================
Someone | facebook | @someone | youtube | linkedin | google     | @tele    | @someins

I have written following query, It works as I expected but I wonder if there's a better way for doing this:

users.Select(p => new UserViewModel {
            Name = p.Name,
            Facebook = p.SocialLinks.FirstOrDefault(x => x.Type == SocialType.FaceBook).Url ?? "#",
            Twitter = p.SocialLinks.FirstOrDefault(x => x.Type == SocialType.Twitter).Url ?? "#",
            Youtube = p.SocialLinks.FirstOrDefault(x => x.Type == SocialType.Youtube).Url ?? "#",
            Linkedin = p.SocialLinks.FirstOrDefault(x => x.Type == SocialType.Linkedin).Url ?? "#",
            Telegram = p.SocialLinks.FirstOrDefault(x => x.Type == SocialType.Telegram).Url ?? "#",
            Instagram = p.SocialLinks.FirstOrDefault(x => x.Type == SocialType.Instagram).Url ?? "#",
}).ToList();

You could use a Dictionary and make properties of UserViewModel to address elements of this dictionary as follows:

public class UserViewModel
{
    private readonly Dictionary<SocialType, string> SocialLinks;

    public UserViewModel(User user)
    {
        SocialLinks = (user.SocialLinks ?? new SocialLink[0])
                      .ToDictionary(x => x.Type, x => x.Url);
        Name = user.Name;
    }

    private string GetUrl(SocialType socialType)
    {
        string url;
        return SocialLinks.TryGetValue(socialType, out url) && url != null ? url : "#";
    }

    public string Name { get; set; }

    public string Facebook
    {
        get { return GetUrl(SocialType.FaceBook); }
        set { SocialLinks[SocialType.FaceBook] = value; }
    }
    public string Twitter
    {
        get { return GetUrl(SocialType.Twitter); }
        set { SocialLinks[SocialType.Twitter] = value; }
    }
    public string Youtube
    {
        get { return GetUrl(SocialType.Youtube); }
        set { SocialLinks[SocialType.Youtube] = value; }
    }

    ...
}

Then your query will looks like:

users.Select(u => new UserViewModel(u)).ToList();


Finding the next number in sequence I have a question to find the N th number in the sequence.

         1
        11
        21
      1211
    111221
    312211
  13112221
1113213211

The next number is obtained by saying out loud the number of digits in the previous number.

  eg. inital number =1
  next number = One 1 = 11
  next number = Two 1 = 21
  next number = One 2 and One 1 = 1211

Here is my code for getting next number:

private static void nextSequence(int N){

        int i=1;

        StringBuffer current = new StringBuffer(); // to populate current number
        StringBuffer last = new StringBuffer();   // to populate next number

        System.out.println("1");

        // populate next number in sequence
        current.append(1);
        current.append(1);

        while(i<=N){

            int begin=1; // pointer to next index
            int start=0; // pointer to start index
            int count=1; // count the number of repeats
            int j=1;     // counts the length of characters counted
            while(j<current.length()){

                if(current.charAt(begin)==current.charAt(start)){
                    count++;     // count the repeats between start and begin
                }else{
                    last.append(count);
                    last.append(current.charAt(start));
                    start=begin;
                    count=1;
                }

                begin++;
                j++;
            }

            last.append(count);
            last.append(current.charAt(start));
            System.out.println(last);
            current=last;
            last = new StringBuffer(); // reset the next seq numbe
            i++;
        }
    }

Is this the correct approach? I am able to generate the said sequence, but I am not sure if this is the most efficient.  Could you please help me correct if my I made any mistakes?

This is one of those times when working backwards helps... but, before we get to that...

Nit Picks

Unless you have a really good reason, do not use StringBuffer. Use StringBuilder instead. There are two times to use StringBuffer, when you are modifying the instance from multiple threads, and when you are using a Matcher.appendReplacement() or Matcher.appendTail(). You are doing Neither, so stick with StringBuilder. It's faster.
You should give values some breathing space... use white-space to make things readable. This is not JavaScript where whitespace impacts download time! Lines like:

while(i<=N){

should be:

while (i <= N) {

Having both the variables begin and start is confusing.
It irks me that you hard-code the first result.... why have:

    // populate next number in sequence
    current.append(1);
    current.append(1);

A Better way.

The actual processing for this is simpler if we do some clever backward-forward logic. I spent some time 'doing it differently'... The trick is that you want to extract a function that does it for a single iteration.... so, let's start with the outer function (so it makes sense):

public static final void loopCompute(String input, int count) {
    System.out.println(input);
    for (int i = 0; i < count; i++) {
        input = compute(input);
        System.out.println(input);
    }
}

I have called it loopCompute, and it takes an input String, and a number of times to iterate. It then loops the number of times, and computes the result, and prints the result. It then prints the new result.

So, what about the compute method?

Well, the trick is to count the sequences of digits, but also, the better trick is to start at the end and work backwards, because we can then use those values to build up the output String efficiently working forwards.

private static final String compute(String source) {
    // allocate two mated arrays, one of the digits, the other of the counts.
    // the worst case is that there will be as many counts as input digits
    // so we budget for the worst case.
    char[] digits = new char[source.length()];
    int[] counts = new int[source.length()];

    int pos = -1;
    // work backwards through the input, storing the results forwards in the arrays.
    for (int i = source.length() - 1; i >= 0; i--) {
        if (pos < 0 || source.charAt(i) != digits[pos]) {
            pos++;
            digits[pos] = source.charAt(i);
        }
        counts[pos]++;
    }

    // now work backwards through the counts, and store them forwards in the StringBuilder.
    StringBuilder sb = new StringBuilder(pos * 2);
    while(pos >= 0) {
        sb.append(counts[pos]).append(digits[pos]);
        pos--;
    }
    return sb.toString();
}


Creating a stack class in C++ Create a class called Stack for storing integers. The data members are
  an integer array for storing the integers and an integer for storing
  the top of stack (tos). Include member functions for initializing tos
  to 0, pushing an element to the stack and for popping an element from
  the stack. The push() function should check for “stack overflow” and
  pop() should check for “stack underflow”.

#include<iostream>
using namespace std;
#define SIZE 50
class Stack {
    int stackArray[SIZE];
    int tos;
public:
    Stack() {
        tos = 0;
    }
    void push(int);
    int pop();
    void set(int);

};

void Stack::push(int value) {

    if (tos < SIZE) {
        stackArray[tos++] = value;
    }
    else
        cout << "Stack overflow" << endl;

}
void Stack::set(int a) {
    tos = a;

}
int Stack::pop() {
    if (tos == 0) {
        cout << "Stack underflow " << endl;
    }
    else {
        return stackArray[--tos];
    }

}
int main() {
    Stack s;
    for (int i = 0; i < 50; i++) {
        s.push(i);
    }
    for (int i = 0; i < 51; i++) {
        cout << s.pop() << " " << i << endl;
    }
}

Overall, this looks pretty good. If it were me, I'd use a more descriptive name for the top of the stack such as topOfStack or just top. tos doesn't really mean anything, so without a comment describing it, I would be confused seeing that in code. (And if you need to write a comment to explain a variable name, you've probably named it incorrectly.)

I'm not sure your set() method works correctly. From the description it sounds like it shouldn't take any arguments and should set tos to 0. But it's possible I misunderstood. If you set the top to some passed-in value, then you have a bunch of junk on the stack between its previous value and the current value, which probably isn't what you want.


Basic C copy file implementation I'm a java programmer by trade and starting to learn C/C++. I'd appreciate any pointers on glaring issues with my code style which is acceptable in java but not in C. I feel like open_files is probably bad in C I'm used to that sort of thing in java to make code easier to scan ( setting *src_file on one line and testing the next)

#include <stdio.h>
#include <string.h>

void get_paths(char *src, char *dest, int buf_size) {
    fprintf(stdout, "Please enter src path:\n");
    fgets(src, buf_size, stdin);
    src[strcspn(src, "\r\n")] = 0;
    fprintf(stdout, "\nPlease enter dest path:\n");
    fgets(dest, buf_size, stdin);
    dest[strcspn(dest, "\r\n")] = 0;
}

int open_files(char *src, char *dest, FILE **src_file, FILE **dest_file) {
    *src_file = fopen(src, "r");
    if(!*src_file) {
        fprintf(stderr, "\n%s is not a valid file or permission denied\n", src);
        return 0;
    }

    *dest_file = fopen(dest, "w");
    if(!*dest_file) {
        fprintf(stderr, "\ncould not open %s for writing, check path exists and you have write permissions\n", dest);
        fclose(*src_file);
        return 0;
    }
    return 1;
}

int cpy(char *cpy_buf, FILE *src_file, FILE *dest_file) {
    size_t num_elements;
    while((num_elements = fread(cpy_buf, sizeof(char), sizeof(cpy_buf), src_file)) > 0) {
        if(fwrite(cpy_buf, sizeof(char), num_elements, dest_file) != num_elements) {
            fprintf(stderr, "\nError while writing to destination \nAborting...");
            return 0;
        }

    }
    return 1;
}

int main() {
    const int buf_size = 256;
    const int cpy_buf_siz = 64;
    char src[buf_size];
    char dest[buf_size];
    char cpy_buf[cpy_buf_siz];
    FILE *src_file;
    FILE *dest_file;

    get_paths(src, dest, buf_size);
    fprintf(stdout, "\nAttempting to copy from %s to %s...\n", src, dest);

    if(!open_files(src, dest, &src_file, &dest_file))
        return -1;

    fprintf(stdout, "\nStarting copy from %s to %s...\n", src, dest);
    int success = cpy(cpy_buf, src_file, dest_file);
    fclose(src_file);
    fclose(dest_file);

    return success ? 0 : 1;
}

You test the return value of fwrite, which is good. However, fread may fail as well. Since fread doesn't distinguish error from end of file, you should call ferror:

    while ((num_elements = fread(....)) > 0) {
        ....
    }
    if (ferror(src)) {
        handle_error
    }

fprintf(stdout), while technically valid, looks strange. A printf suffices.
cpy_buf doesn't belong to main. Define it directly in cpy.
Prefer passing file names via command line arguments.
The "Aborting..." message doesn't have a terminating newline, and the next prompt will be printed on the same line (on any shell except cmd.exe). Make a habit to print a newline at the end of the message.


Adding groups to ListView depending on the data in a list I have written simple code which loops through a list object and create groups in ListView depending on the data in the list. I have a list object of type List<Assignment> where Assignment is a class as follows.

public class Assignment
{
    public TimeSpan Time { get; set; }
    public string Type { get; set; }
    public string Description { get; set; }
    public int ImageIndex { get; set; }
}

Now, the list object have several assignments with different TimeSpan. My job is to group together all the Assignments with similar TimeSpan into one group and add this group to the ListView. The following code works perfectly fine for me. Still, I would love to know if there is better way of doing this.

The list passed into the function is sorted by TimeSpan.

private void FeedDataToPane(List<Assignment> assignmentList)
{
    ListViewGroup _listViewGrp;
    ListViewItem _listViewItem;

    MonitorPane.BeginUpdate();
    MonitorPane.Groups.Clear();
    MonitorPane.SmallImageList = imageList;

    for (int i = 0; i < assignmentList.Count; i++)
    {
        if (i == 0)
        {
            _listViewGrp = new ListViewGroup(assignmentList[i].Time.ToString());
            MonitorPane.Groups.Add(_listViewGrp);

            _listViewItem = new ListViewItem(assignmentList[i].Type, assignmentList[i].ImageIndex, _listViewGrp);
            _listViewItem.SubItems.Add(assignmentList[i].Description);
            MonitorPane.Items.Add(_listViewItem);
        }
        else
        {
            if (assignmentList[i].Time.Equals(assignmentList[i - 1].Time))
            {
                _listViewItem = new ListViewItem(assignmentList[i].Type, assignmentList[i].ImageIndex, _listViewGrp);
                _listViewItem.SubItems.Add(assignmentList[i].Description);
                MonitorPane.Items.Add(_listViewItem);
            }
            else
            {
                _listViewGrp = new ListViewGroup(assignmentList[i].Time.ToString());
                MonitorPane.Groups.Add(_listViewGrp);

                _listViewItem = new ListViewItem(assignmentList[i].Type, assignmentList[i].ImageIndex, _listViewGrp);
                _listViewItem.SubItems.Add(assignmentList[i].Description);
                MonitorPane.Items.Add(_listViewItem);
            }
        }
    }

    MonitorPane.EndUpdate();
}

It's possible with the help of LINQ to reduce your code to this (assuming, of course, the version of .NET you're using will allow it):

private void FeedDataToPane(IEnumerable<Assignment> assignmentList)
{
    MonitorPane.BeginUpdate();
    MonitorPane.Groups.Clear();
    MonitorPane.SmallImageList = imageList;

    assignmentList
        .GroupBy(assignment => assignment.Time, CreateListViewGroup)
        .ToList();

    MonitorPane.EndUpdate();
}

private static ListViewGroup CreateListViewGroup(TimeSpan time, IEnumerable<Assignment> assignments)
{
    var group = new ListViewGroup(time.ToString());
    MonitorPane.Groups.Add(group);

    foreach (var assignment in assignments)
    {
        var item = new ListViewItem(assignment.Type, assignment.ImageIndex, group);
        item.SubItems.Add(assignment.Description);
        MonitorPane.Items.Add(item);
    }

    return group;
}


I want to check neighborhoods of cell in two dimensional array in C++ I want to write a function which will return all possible/valid 'X' moves around my Target cell depending on it position inside the following two dimensional array.

X X X
X T X
X X X

But my 2 concerns are:

the performance because my "game field" will be 1000x1000 cells and each cell must be checked in the game turn. So creating 1,000,000 vectors (in one turn) is a bad idea? Maybe I should use a different data structure for it?
logic inside the function is not too complicated?

enum dir {topL, top, topR, l, r, botL, bot, botR};

std::vector<dir> neighborhood(int x, int y, int size)
{
    std::vector<dir> direction;
    direction.clear();

    if(x < size -1){direction.push_back(bot);}
    if(y < size -1){direction.push_back(r);}
    if(x < size -1 && y < size -1){direction.push_back(botR);}
    if(x > 0){direction.push_back(top);}
    if(y > 0){direction.push_back(l);}
    if(x < size -1 && y != 0){direction.push_back(botL);}
    if(x > 0 && y < size -1){direction.push_back(topR);}
    if(x > 0 && y > 0){direction.push_back(topL);}
    return direction;
}

You can use 8 bit memory (char in c/c++) to precompute possible direction of movement for each of the cells and use bitwise operation to improve performance:

//bit1: top direction, bit2: topright directon, bit3: rigth, and so on....
std::array<char, 1000000> c;    //array, this will be our playground
c.assign(255);                  //to set all ones in all cells
c[0] = c[0] & 0x38;             //top left cell representing 00111000
c[999] = c[999] & 0xE;          //top right cell
c[900000] = c[900000] & 0xE0;   //bottom left cell
c[999999] = c[999999] & 0x83;   //bottom right cell

for(int i=1; i<999; i++)               //all top edge cells except corners
   c[i] &= 0x3E;
for(int i=1999; i<999999; i += 1000)   //all right edge cells except corners
   c[i] &= 0x8F
for(int i=9999000; i<999999; i ++)     //all bottom edge cells except corners
   c[i] &= 0xE3
for(int i=1000; i<999000; i += 1000)   //all left edge cells except corners
   c[i] &= 0xF8

then you can use these hex values to quickly determine if your cell is at corners, edges or in the middle
    example:

  if(! c[i] ^ 0x38){
       //this cell is left corner
    }

Because bitwise operators are very fast, you will see great improvement in performance.


SKlearn automate data pre treatment I want to make a simple wrapper for sklearn models. The idea is that the wrapper automatically takes care of factors (columns of type "object") replacing them with the average value of the target, while preserving the syntax of sklearn models.

If the factor is too scarce, it is replaced by the overall average value of the target. It seems simple but issues arise when a factor is in the test set and was not in the train set. I came up with the following solution, that looks really awkward to me.

class ModelEmbedder :

    def __init__(self, model, rare_threshold) :
        self.model = model
        self.means = {}
        self.rare_threshold = rare_threshold
        self.train = None
        self.origin_train = None
        self.average = 0

    def fit(self,train,target):
        self.origin_train = train.copy().fillna(-1)
        self.train = train.copy()
        self.train = self.train.fillna(-1)
        self.train['target'] = target
        self.average = target.mean()

        for feat in train.columns:
            if feat != 'target' :
                if self.train[feat].dtype=='object' :
                    self.train.loc[self.train[feat].value_counts()[self.train[feat]].values <  self.rare_threshold, feat] = "RARE"
                    self.origin_train.loc[self.origin_train[feat].value_counts()[self.origin_train[feat]].values <  self.rare_threshold, feat] = "RARE"
                    self.means[feat] = self.train.groupby([feat])['target'].mean()
                    self.means[feat]["RARE"] = self.average

                    self.train[feat] = self.train[feat].replace(self.means[feat], inplace=False)

        del self.train['target']

        self.model.fit(self.train,target)

    def _pre_treat_test(self,test) :
        test = test.copy()
        test = test.fillna(-1)

        for feat in self.origin_train.columns:
            if self.origin_train[feat].dtype=='object' :
                test.loc[self.origin_train[feat].value_counts()[self.origin_train[feat]].values  <  self.rare_threshold, feat] = "RARE"

                criterion = ~test[feat].isin(set(self.origin_train[feat]))

                test.loc[criterion,feat] = self.average

                test[feat]  = test[feat].replace(self.means[feat], inplace=False)

        return test

    def predict_proba(self,test) :
        test = self._pre_treat_test(test)
        return self.model.predict_proba(test)

    def get_params(self, deep = True):
        return self.model.get_params(deep)

Then, every model can be wrapped:

rf = ensemble.ExtraTreesClassifier(n_jobs=7,
    n_estimators = n_estimators,
    random_state = 11)

rf_embedded = model_embedder.ModelEmbedder(rf,10)

and sent to a cross-validation loop or any pipeline.

A few observations:

You are adding spaces before most of your : throughout, which reads weird. I'd remove this as I've never seen any other Python code that does this
I would check out Pep8 as a way to clean up a variety of formatting things, as it is the more or less accepted Python formatting standard

This will show things like lacking spaces around operators, too many line breaks, etc

Also, I am not sure how you are using underscores here. Generally, underscores signify private members/methods in Python and it looks like you have quite a few places you could make your member variables private.

Also this could be a bit more clear:

self.train.loc[self.train[feat].value_counts()[self.train[feat]].values <  self.rare_threshold, feat] = "RARE"

You might consider some intermediate steps here to help with readability. Even something as simple as:

val = self.train[feat].value_counts()[self.train[feat]].values <  self.rare_threshold
self.train.loc[val, feat] = "RARE"

is more clear. Trying to read that many nested dictionary lookups is not simple.


Android text to speech app Since I am trying to learn many programming languages, I created this simple Android app. What it does basically is taking a sentence phrase from an EditText ed and translate it to speech when you click the start button.
I used onActivityResult to prevent app from crashing when the EditText becomes empty.
I am sensing that this code, while working perfectly, is far from perfect.
import android.support.v7.app.AppCompatActivity;
import java.util.Locale;
import android.content.Intent;
import android.os.Bundle;
import android.speech.tts.TextToSpeech;
import android.speech.tts.TextToSpeech.OnInitListener;
import android.view.View;
import android.widget.Button;
import android.widget.EditText;
import android.widget.Toast;

public class MainActivity extends AppCompatActivity implements OnInitListener {

private TextToSpeech myTTS;
private EditText ed;
private String phrase;

@Override
protected void onCreate(Bundle savedInstanceState) {

    super.onCreate(savedInstanceState);
    setContentView(R.layout.activity_main);

    ed = findViewById(R.id.editText);

    Button startButton = findViewById(R.id.start_button);
    startButton.setOnClickListener(new View.OnClickListener() {
        @Override
        public void onClick(View arg0) {

            Intent checkIntent = new Intent();
            checkIntent.setAction(TextToSpeech.Engine.ACTION_CHECK_TTS_DATA);
            startActivityForResult(checkIntent, 1);

            phrase = ed.getText().toString();

        }
    });
}

protected void onActivityResult(int requestCode, int resultCode, Intent data) {
    if (requestCode == 1) {

        if (resultCode == TextToSpeech.Engine.CHECK_VOICE_DATA_PASS) {
            myTTS = new TextToSpeech(this, this);
            myTTS.setLanguage(Locale.US);
        } else {
            // TTS data not yet loaded, try to install it
            Intent ttsLoadIntent = new Intent();
            ttsLoadIntent.setAction(TextToSpeech.Engine.ACTION_INSTALL_TTS_DATA);
            startActivity(ttsLoadIntent);
        }
    }
}

public void onInit(int status) {
    if (status == TextToSpeech.SUCCESS) {
        myTTS.speak(phrase, TextToSpeech.QUEUE_FLUSH, null);
    } else if (status == TextToSpeech.ERROR) {
        myTTS.shutdown();
    }
}

}

First of all better use clear variable names.
Coming from android I can guess what an 'ed' is. But for everybody it is always better to name it properly like :
private EditText editText;
editText = findViewById(R.id.editText);

or in this case maybe even
editTextTTS;

or something similar.
Same with myTTS, I personally would also name that :
private TextToSpeech textToSpeech;

You are creating a TextToSpeech Object on every click, which is absolutely unneccessary, create it once when starting the application, and reuse it. So this piece of code belongs in your onCreate() not your onClick() :
Intent checkIntent = new Intent();
checkIntent.setAction(TextToSpeech.Engine.ACTION_CHECK_TTS_DATA);
startActivityForResult(checkIntent, 1);

That way the TTS will be ready before the user can click the button and is only initialised once.
You are getting the value of your editText AFTER you startActivityForResult().
This does not seem to be the logical order, although if you remove the code above from your onClick(), that also solves this problem.
You have a magic number 1 in your startActivityForResult() method, better use a final value with a proper name like :
private final int TTS_REQUEST_CODE = 1;

makes this also look way better :
startActivityForResult(checkIntent, TTS_REQUEST_CODE);

if (requestCode == TTS_REQUEST_CODE)

In case the application becomes larger and more complex this will be way more secure and readable since the value is final and cannot be tampered with.
There is no information to the user wether the TextToSpeech is installed and ready or not, I would suggest showing a Toast or a Snackbar to let the user know what is happening, especially in case of
status == TextToSpeech.ERROR

You might even want to smooth exit the application in this case.
This app will only please Users with US English language. Why not make it get the Locale from the Users device, to set the proper language dynamically :
myTTS.setLanguage(Locale.getDefault());

The onInit() method should only be called once when initialising the TTS Object, so your
myTTS.speak(phrase, TextToSpeech.QUEUE_FLUSH, null);

is misplaced there. This code should go into your onClick() method along with a check that the editText is not empty :
public void onClick(View arg0) {

    final String phrase = ed.getText().toString();
    if(!phrase.isEmpty()) {
        myTTS.speak(phrase, TextToSpeech.QUEUE_FLUSH, null);
    } else {
        // Toast or Snackbar to tell the user Edittext is empty
    }

This will also convert
private String phrase;

to a local final variable, which is always better than having an unneccessary global variable.


Display a simulation using Tkinter I wrote a module to simulate physics of 2D elastic balls and the community helped me to improve it on this post.

Now I implemented a GUI using Tkinter to display the simulation in a window.

I'm a beginner in programming GUI and I don't know if my script can be more efficient and/or simpler.

Indeed, I'm not really satisfied by my display function because it includes definitions of other functions dedicated to the buttons commands.

Moreover, when I push the start button twice, I also need to push the pause button twice to stop the simulation. I don't understand this behaviour !

Import modules

import Tkinter as tk
import solver

You'll find the solver module here. This isn't the object of this post. If you've comments or remarks about it, post them on this post I spoke above.

Surrounding functions

def _create_circle(self, x, y, r, **kwargs):
    """Create a circle

    x the abscissa of centre
    y the ordinate of centre
    r the radius of circle
    **kwargs optional arguments
    return the drawing of a circle
    """
    return self.create_oval(x-r, y-r, x+r, y+r, **kwargs)
tk.Canvas.create_circle = _create_circle

def _coords_circle(self, target, x, y, r, **kwargs):
    """Define a circle

    target the circle object
    x the abscissa of centre
    y the ordinate of centre
    r the radius of circle
    **kwargs optional arguments
    return the circle drawing with updated coordinates
    """
    return self.coords(target, x-r, y-r, x+r, y+r, **kwargs)
tk.Canvas.coords_circle = _coords_circle

def create(balls, canvas):
    """Create a drawing item for each solver.Ball object

    balls the list of solver.Ball objects
    canvas the Tkinter.Canvas oject
    return a dictionary with solver.Ball objects as keys and their circle drawings as items
    """
    return {ball: canvas.create_circle(ball.position[0], ball.position[1], ball.radius, fill="white") for ball in balls}

def update(drawing, canvas, step, size):
    """Update the drawing items for a time step

    drawing the dictionary of drawing items
    canvas the Tkinter.Canvas oject
    step the time step
    size the medium size
    """
    balls = drawing.keys()
    solver.solve_step(balls, step, size)
    for ball in balls:
        canvas.coords_circle(drawing[ball], ball.position[0], ball.position[1], ball.radius)
    canvas.update()

display function

def display(balls, step, size):
    """Display the simulation

    balls the list of solver.Ball objects
    step the time step
    size the medium size
    """
    # Instanciate the window, canvas and circle objects
    window = tk.Tk()
    window.poll = True
    canvas = tk.Canvas(window, width=size, height=size, bg="black")
    canvas.pack()
    canvas.focus_set()
    drawing = create(balls, canvas)
    # Define functions to launch and stop the simulation
    def animate():
        """Animate the drawing items"""
        if window.poll:
            update(drawing, canvas, step, size)
            window.after(0, animate)
        else:
            window.poll = True
    def stop():
        """Stop the animation"""
        window.poll = False
    # Define the buttons used to launch and stop the simulation
    start_button = tk.Button(window, text="Start", command=animate)
    stop_button = tk.Button(window, text="Pause", command=stop)
    start_button.pack()
    stop_button.pack()
    # GUI loop
    window.mainloop()

Unit test of display function

# Test this module
if __name__ == "__main__":
    balls = [solver.Ball(20., 20., [40.,40.], [5.,5.]), solver.Ball(10., 10., [480.,480.], [-15.,-15.]), solver.Ball(15., 15., [30.,470.], [10.,-10.])]
    size = 500.
    step = 0.1
    display(balls, step, size)

As I didn't get any answer, I propose the following improvements.

First I didn't change the functions create_circle and coords_circle.

I found a more satisfying solution for the display function. Indeed, I create a class Display and implement the former display function in its __init__ method.

class Display:
    """Define the window used to display a simulation"""

    def __init__(self, balls, step, size):
        """Initialize and launch the display"""
        self.balls = balls
        self.step = step
        self.size = size

        self.window = tk.Tk()
        self.canvas = tk.Canvas(self.window, width=self.size, height=self.size, bg="black")
        self.canvas.pack()
        self.canvas.focus_set()
        self.drawing = self.create()
        self.started = False

        start_button = tk.Button(self.window, text="Start", command=self.start)
        stop_button = tk.Button(self.window, text="Pause", command=self.stop)
        start_button.pack()
        stop_button.pack()

        self.window.mainloop()

In this way, I can define the animate and stop functions as methods of the class Display rather than inside __init__.

The objects balls, step, size, window, canvas, drawing, start_button and stop_button become attributes of the class Display. So I don't need to put them in arguments of create and update methods.

def create(self):
    """Create a drawing item for each solver.Ball object

    return a dictionary with solver.Ball objects as keys and their circle drawings as items
    """
    return {ball: self.canvas.create_circle(ball.position[0], ball.position[1], ball.radius, fill="white") for ball in self.balls}

def update(self):
    """Update the drawing items for a time step"""
    solver.solve_step(self.balls, self.step, self.size)
    for ball in self.balls:
        self.canvas.coords_circle(self.drawing[ball], ball.position[0], ball.position[1], ball.radius)
    self.canvas.update()

Moreover, the start button calls the start method that calls the animate method only if the value of started is False. Thus, we don't have the bad behaviour raised in the question (need to push the pause button twice after pushing the start button twice).

def start(self):
    """Start the animation"""
    if not self.started:
        self.started = True
        self.animate()

def animate(self):
    """Animate the drawing items"""
    if self.started:
        self.update()
        self.window.after(0, self.animate)

def stop(self):
    """Stop the animation"""
    self.started = False

You'll find the complete code here.


Will multi-threading or other method make my program run faster? I didn't use multi-threading so far as I didn't need to. But as far as I've read, implementing them will make my program slightly faster than it actually is.

from validate_email import validate_email
import os

# the program is reading each line from "emails.txt" and after it checks each email it will remove duplicates and sort the godd / bad emails

def verify_emails(all_emails_file, all_good_emails_file, all_bad_emails_file):
    with open(all_emails_file) as f: all_emails = f.readlines()

    rs_emails = [elem.strip('\n') for elem in all_emails]
    rs_emails_set = set(rs_emails)  # remove duplicates

    good_emails_file, bad_emails_file = open(all_good_emails_file, 'w+'), open(all_bad_emails_file, 'w+')

    for email in rs_emails_set:
        if validate_email(email, verify=True):
            print >> good_emails_file, email
        else:
            print >> bad_emails_file, email

if __name__ == "__main__":
    clear = lambda: os.system('cls')
    clear()
    try:
        verify_emails("emails.txt", "good_emails.txt", "bad_emails.txt")
    except:
        print "\n\nFile with emails could not be found. Please create emails.txt and run the program again\n\n"

My code is perfectly functional, but when it handles big files ( > 2k rows ) it runs really slow. I'd like to take the best out of it and make it as faster as possible using multi-threading or any other methods.

I'd like when answering, if possible, somebody to explain me whether using multi-threading will help me optimize the program or not. More, I'd like somebody to also explain from his / her experience how can I optimize my code

As stated in the comments, multi-threading might not be what you are looking for.

I think there is room for improvement on the way you read your file. Currently:

you read the whole file into a string all_emails = f.readlines()
you remove duplicates rs_emails_set = set(rs_emails)  # remove duplicates
and you read every element of this array for email in rs_emails_set:

Reading this comment, I strongly recommend you to test the following:

processed_emails = set()
for email in f:
  if email not in processed_emails:
    # validate email or not
    processed_emails.add(email)

Instead of immediately writing the good and bad emails files, you could store those into 2 lists and write them at once at the very end (removing some I/O with the filesystem as well)


Better way of handling isset/!empty? $controller = new UserController();

if (isset($_POST['submitform']))
{
    $validated = false;

    $inputs = array (
        'username', 'email', 'password',
        'repassword', 'password_f', 'repassword_f',
        'display'
    );

    $i = 0;

    foreach ($inputs as $key)
    {
        if (isset($_POST[$key]) && !empty($_POST[$key]))
        {
            $i++;
            if ((int)$i == count($inputs))
            {
                $validated = true;
                break;
            }
            else
            {
                $controller->error = "";
                break;
            }
        }
    }

    if ($validated)
    {
        echo 2;
    }
}

So let's overview the code.

First we are checking if the form was submitted.

Then we are creating a new boolean $validated and setting it false by default.

Then we created our array with the POST names.

Now the checking part, setting variable int i to 0 by default.
Now we are looping through the array elements.

Checking if the current index was set or not empty, if yes, add +1 to int $i.

Once int $i hits the number of counted elements of our array, set $validated to true, and break out of the array, else parse error and break out.

Then we are checking, if bool $validated is true, then echo 2.

Is it a good way of doing this? Is there any cleaner/better way?

I'd do this instead:

function validateForm() {
    if (!isset($_POST['submitform']))
        return false; // Maybe $controller->error = "Something"; as well?
    $inputs = array (
        'username', 'email', 'password',
        'repassword', 'password_f', 'repassword_f',
        'display'
    );
    foreach ($inputs as $key)
        if (empty($_POST[$key]))
        {
            global $controller;
            $controller->error = "";
            return false;
        }
    return true;
}
...
if (valudateForm()) {
    echo 2;
} else {
    // Form data not ok
}

Seems cleaner, no flags, no counting array elements, validation has its own function,...


Cythonic Laplacian solving on a 3D regular grid I have followed to official Cython for NumPy users to make a Laplacian solver on a 3D regular grid with Dirichlet conditions.

Right now I'm happy with the results it returns, but I'm wondering if there was a more efficient way to code this, especially since the tutorial I have followed is about 10 years old. I have very limited knowledge of C and I'm not sure if the data types (ctypedef) are properly chosen here and/or would have a significant impact on performance. I have tried types my indexes variables as unsigned int in different ways but I ended up giving up as I'm not really sure if it is relevant.

Also, I'm happy with any general coding style and good practices advice.

The Python code that calls my Cython function is inside a class and looks like this:

def _solve_laplacian(self):
    init = np.zeros_like(self.domain, float)
    init[np.logical_not(self.border)] = 1
    log.info("Solving Laplacian...")
    laplace_grid, iterations, max_error = laplace_solver(
        self.domain_idx_i,
        self.domain_idx_j,
        self.domain_idx_k,
        init,
        self.laplace_tolerance,
        self.laplace_max_iter,
        self.spacing)
    log.debug(
        f"Laplacian: {iterations} iterations, max_error = {max_error}")
    self.cropped_laplace_grid = laplace_grid
    self.laplacian_iterations = iterations
    self.laplacian_max_error = max_error

and here is the actual cython function:

import numpy as np
cimport numpy as np
cimport cython

ctypedef np.float64_t DTYPE_FLOAT
ctypedef np.int_t DTYPE_INT

# dirty fix to get rid of annoying warnings
np.seterr(divide='ignore', invalid='ignore')

@cython.boundscheck(False)
@cython.wraparound(False)
def solve_3D(np.ndarray[DTYPE_INT, ndim=1] domain_idx_i,
             np.ndarray[DTYPE_INT, ndim=1] domain_idx_j,
             np.ndarray[DTYPE_INT, ndim=1] domain_idx_k,
             np.ndarray[DTYPE_FLOAT, ndim=3] init,
             DTYPE_FLOAT tolerance,
             DTYPE_INT max_iterations,
             np.ndarray[DTYPE_FLOAT, ndim=1] spacing):
    cdef np.ndarray[DTYPE_FLOAT, ndim=3] laplace_grid = np.copy(init)
    cdef np.ndarray[DTYPE_FLOAT, ndim=3] prev_laplace_grid = np.copy(
        laplace_grid)
    cdef DTYPE_INT iteration = 0
    cdef DTYPE_FLOAT max_error = 1.0
    cdef DTYPE_INT n_points = len(domain_idx_i)
    cdef DTYPE_INT i, j, k, n
    cdef DTYPE_FLOAT error
    cdef DTYPE_FLOAT value
    cdef DTYPE_FLOAT hi, hj, hk, hi2, hj2, hk2, factor

    hi, hj, hk = spacing
    hi2, hj2, hk2 = spacing ** 2

    factor = (hi2 * hj2 * hk2) / (2 * (hi2 * hj2 + hi2 * hk2 + hj2 * hk2))

    while max_error > tolerance and iteration < max_iterations:
        iteration += 1
        prev_laplace_grid[:] = laplace_grid
        for n in range(n_points):
            i = domain_idx_i[n]
            j = domain_idx_j[n]
            k = domain_idx_k[n]
            value = \
                (
                    (laplace_grid[i + 1, j, k] +
                     laplace_grid[i - 1, j, k]) / hi2 +
                    (laplace_grid[i, j + 1, k] +
                     laplace_grid[i, j - 1, k]) / hj2 +
                    (laplace_grid[i, j, k - 1] +
                     laplace_grid[i, j, k + 1]) / hk2
                ) * factor
            laplace_grid[i, j, k] = value
        max_error = np.nanmax(
            np.abs(
                (prev_laplace_grid[domain_idx_i, domain_idx_j, domain_idx_k] -
                 laplace_grid[domain_idx_i, domain_idx_j, domain_idx_k]) /
                prev_laplace_grid[domain_idx_i, domain_idx_j, domain_idx_k]))
    return laplace_grid, iteration, max_error

If you're interested in the complete Python package this is part of, it is available here and is quite ugly right now. I'm trying to improve it so I'm not ashamed of it anymore.

I was helped on the #scipy IRC channel by user @ngoldbaum and could substantially speed up my code by:

Avoiding to use any numpy methods in the main loop
Using OpenMP parallelization
Using true C division as there is no risk of division by zero
Avoiding to store the whole previous iteration values

…and some other minor stuff.

Here's what my solver looks like now:

import numpy as np
cimport numpy as np
cimport cython
from cython.parallel import prange
from libc.math cimport fabs

ctypedef np.float64_t DTYPE_FLOAT
ctypedef np.int_t DTYPE_INT

# dirty fix to get rid of annoying warnings
np.seterr(divide='ignore', invalid='ignore')

@cython.boundscheck(False)
@cython.wraparound(False)
cdef bint has_converged(np.ndarray[DTYPE_FLOAT, ndim=1] errors,
                        DTYPE_INT n,
                        DTYPE_FLOAT tolerance):
    cdef bint res = True
    cdef DTYPE_INT i
    for i in prange(n, nogil=True):  # is prange useful here ?
        if errors[i] > tolerance:
            res = False
            break
    return res

@cython.boundscheck(False)
@cython.wraparound(False)
@cython.cdivision(True)
def solve_3D(np.ndarray[DTYPE_INT, ndim=1] domain_idx_i,
             np.ndarray[DTYPE_INT, ndim=1] domain_idx_j,
             np.ndarray[DTYPE_INT, ndim=1] domain_idx_k,
             np.ndarray[DTYPE_FLOAT, ndim=3] init,
             DTYPE_FLOAT tolerance,
             DTYPE_INT max_iterations,
             np.ndarray[DTYPE_FLOAT, ndim=1] spacing):
    cdef np.ndarray[DTYPE_FLOAT, ndim=3] laplace_grid = np.copy(init)
    cdef DTYPE_INT iteration = 0
    cdef DTYPE_INT n_points = len(domain_idx_i)
    cdef DTYPE_INT i, j, k, n
    cdef DTYPE_FLOAT value
    cdef DTYPE_FLOAT hi, hj, hk, hi2, hj2, hk2, factor, prev_value
    cdef np.ndarray[DTYPE_FLOAT, ndim=1] errors = np.zeros(n_points,
                                                           np.float64)
    cdef bint convergence = False

    hi, hj, hk = spacing
    hi2, hj2, hk2 = spacing ** 2

    factor = (hi2 * hj2 * hk2) / (2 * (hi2 * hj2 + hi2 * hk2 + hj2 * hk2))

    while not convergence and iteration < max_iterations:
        iteration += 1
        for n in prange(n_points, nogil=True):
            i = domain_idx_i[n]
            j = domain_idx_j[n]
            k = domain_idx_k[n]
            value = ((laplace_grid[i + 1, j, k] +
                      laplace_grid[i - 1, j, k]) / hi2 +
                     (laplace_grid[i, j + 1, k] +
                      laplace_grid[i, j - 1, k]) / hj2 +
                     (laplace_grid[i, j, k - 1] +
                      laplace_grid[i, j, k + 1]) / hk2) * factor
            prev_value = laplace_grid[i, j, k]
            laplace_grid[i, j, k] = value
            errors[n] = fabs((prev_value - value) / prev_value)

        if iteration == 1:
            convergence = False
        else:
            convergence = has_converged(errors, n, tolerance)

    return laplace_grid, iteration, np.nanmax(errors)

I think it could be a bit better if errors were checked "on the fly", removing the second for loop every while iteration. However, I did not find out to do that efficiently in a multithread prange… maybe it's not worth it!


Show a ProgressDialog and send an SMS in the background I have written a procedure that

shows a ProgressDialog
sends an SMS
updates the View from another thread
hides the ProgressDialog

But it seems to me that this is not the best way to do this. I would like to hear advice and my mistakes. It seems to me that it is possible to optimize the code for quick work procedures.

public boolean sendCommand(commandXDev command, final String uniValue) {
    SharedPreferences prefs = this.getSharedPreferences("serjaru.bsh.xdev2", Context.MODE_PRIVATE);
    final String phoneNumber = prefs.getString("phonenumber", "");
    int gateway = prefs.getInt("gateway", 0);

    if (gateway == 0) {

        if (command == commandXDev.Get) {
            final ProgressDialog progress = ProgressDialog.show(Relays.this, "Please wait", "Sending SMS...", true);
            progress.setCancelable(true);

            final Handler handler = new Handler();
            Runnable runnable = new Runnable() {

                public void run() {
                    if (myPhone.sendSMSMessage("+" + phoneNumber, ImeiPhone + "#get")) {
                        handler.post(new Runnable() {
                            public void run() {
                                statusLabel.setText("SMS sended!");
                            }
                        });
                    } else {
                        handler.post(new Runnable() {
                            public void run() {
                                statusLabel.setText("Error. SMS not sended.");
                            }
                        });
                    }
                    progress.dismiss();
                }
            };
            new Thread(runnable).start();
        }
    }
}

What you are trying to achieve is such a common pattern (run some command in separate thread and then modify the UI with the result) that Android has a class just for that: AsyncTask.  There is also a small example in the dev guide and I'm sure all Android books must have a full section, if not a full chapter, about AsyncTask.

Be very careful about Java conventions.  When you don't respect them, the code is quite confusing for Java developers.

Class names always start a capital letter, so CommandXDev instead of commandXDev.
commandXDev.Get.  I'm not quite sure what that is.  In Java, if you write a getter method, you always call it get followed by what you are getting, i.e. getName(), getId(), getSharedPreferences(), etc.  Your commandXDev.Get seems to be a static getter, which is best avoided.  I'm not sure what you are trying to do.  Maybe look into the singleton pattern.

Minor point: in English it is "sent" instead of "sended".  It does not make sense, but it is how it is.