Split an array into m subarrays such that the largest subarray sum is minimized This is a Leetcode problem -

  Given an array which consists of non-negative integers and an integer
  m, you can split the array into m non-empty continuous subarrays.
  Write an algorithm to minimize the largest sum among these m
  subarrays.

  Note -

  If n is the length of the array, assume the following constraints
  are satisfied:

  1 ≤ n ≤ 1000
  1 ≤ m ≤ min(50, n)

Here is my solution to this challenge -

class Solution(object):

    def __init__(self, nums, m):
        self.nums = nums
        self.m = m
    def split_array(self, nums, m):
        """
        :type nums: List[int]
        :type m: int
        :rtype: int
        """
        min_res = max(nums)
        max_res = sum(nums)
        low, high = min_res, max_res
        while low + 1 < high:
            mid = low + (high - low)//2
            if self.is_valid(nums, m, mid):
                high = mid
            else:
                low = mid
        if self.is_valid(nums, m, low):
            return low
        return high

    def is_valid(self, nums, m, n):
        count, current = 1, 0
        for i in range(len(nums)):
            if nums[i] > n:
                return False
            current += nums[i]
            if current > n:
                current = nums[i]
                count += 1
        if count > m:
            return False
        return True

Here, I just optimize the binary search method (technique used to search an element in a sorted list), change the condition of low <= high to low + 1 < high and mid = low + (high - low) / 2 in case low + high is larger than the max int.

Here is an example input/output -

output = Solution([7,2,5,10,8], 2)
print(output.split_array([7,2,5,10,8], 2))
>>> 18

Explanation -

There are four ways to split nums into two subarrays.
The best way is to split it into [7,2,5] and[10,8],
where the largest sum among the two subarrays is only 18.

Here is the time for this output -

%timeit output.split_array([7,2,5,10,8], 2)
12.7 µs ± 512 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)

So, I would like to know whether I could make this program shorter and more efficient.

(Sort of) Unnecessary OOP

A lot of things like LeetCode will require that the solution is in a class for likely submission reasons, but if we concern ourselves with only the task at hand, I would recommend you get rid of the class and just use functions for this. (And if get rid of the unnecessary class, you'll shorten the code, although, I wouldn't encourage you to be preoccupied with shortening your code.)

Also, if you had to keep the class (for whatever reason), observe:

def __init__(self, nums, m):
    self.nums = nums
    self.m = m

When you pass parameters like this, you can reuse them over and over again. They are sort of like pseudo-global variables. So you would need to do:

def is_valid(self, nums, m, n):

it would just be:

def is_valid(self):

and (in general) you would access self.m and self.n.

Iterate over the iterables, not over the len of the iterable.

Unless you are mutating nums (which I don't believe you are) the more idiomatic way of doing this is iterating through the iterable so:

for i in range(len(nums)):

becomes:

for num in nums:

and instances of nums[i] are replaced by num, for example if nums[i] > n: becomes if num > n:.

Side note: if you were to need the value i and nums[i], you might want to consider utilizing enumerate if you need both.


Preg_match pattern for user input filtering My users may need to supply a 'disease category' to my site. I need to let them use all alphanumeric characters, hyphens and single quotes. Would someone review this to see if they feel it's reasonable safe?  I'm very happy for suggestions to make it tighter,

      $disease_category = "some user input";
      $reg = "#[^'a-z0-9\s-]#i";
      preg_match($reg,$disease_category);

Well if having an underscore in your line could be acceptable, then

$reg = "#[^\w\s-']#i";

could be used. If you cannot have underscores, then

$reg = "#[^a-z\d\s-']#i";

could be correct.

A conditional using preg_match would be safe with this.


Golang function to clean a string of scripts I am trying to make an efficient algorithm for removing script tags from an HTML string. Can someone point out any flaws in this? This seems to be the best I could think of.

func removeScripts(s string) string {
    startingScriptTag := "<script"
    endingScriptTag := "</script>"

    var script string

    for {
        startingScriptTagIndex := strings.Index(s, startingScriptTag)
        endingScriptTagIndex := strings.Index(s, endingScriptTag)

        if startingScriptTagIndex > -1 && endingScriptTagIndex > -1 {
            script = s[startingScriptTagIndex:endingScriptTagIndex + len(endingScriptTag)]
            s = strings.Replace(s, script, "", 1)
            continue
        }

        break
    }

    return s
}

As per usual, I'd say the best way to reliably get rid of any script tags in an HTML string, is to use a parser. HTML is a bit too complex to consume using your standard string functions and regular expressions. It's a hierarchical language, best processed as such. Thankfully, golang has a package for this, and it's fantastically easy to remove script tags:

import (
    "bytes"
    "fmt"
    "log"
    "string"

    "golang.org/x/net/html" // go get -u golang.org etc...
)

func main() {
    doc, err := html.Parse(strings.NewReader(htmlString))
    if err != nil {
        log.Fatal(err)
    }
    removeScript(doc)
    buf := bytes.NewBuffer([]bytes{})
    if err := html.Render(buf, doc); err != nil {
        log.Fatal(err)
    }
    fmt.Println(buf.String())
}

func removeScript(n *html.Node) {
    // if note is script tag
    if n.Type == html.ElementNode && n.Data == "script" {
        n.Parent.RemoveChild(n)
        return // script tag is gone...
    }
    // traverse DOM
    for c := n.FirstChild; c != nil; c = c.NextSibling {
        removeScript(c)
    }
}

The use of n.Data is not a typo BTW. The field name is a bit unfortunate, but as the doc pages state:

  A Node consists of a NodeType and some Data (tag name for element nodes, content for text)

This code was not tested. It's loosely based on the parse example in the official godoc pages.

Although not relevant in this case, it is worth looking into the tokenizer API, too. It is a lower-level api, that can help you process an HTML stream (eg parsing/validating a large file in a stream). You can use it to check how many script tags there are, for example:

tokenizer := html.NewTokenizer(strings.NewReader(htmlString))
tagCount := 0
for {
    tt := z.Next()
    switch tt {
    case ErrorToken:
        return z.Err()
    //case TextToken: ignore, we don't need it here
    case StartTagToken //, EndTagToken: ignore end-tags to make life easier
        tn, _ := z.TagName()
        if string(tn) == "script"  {
            tagCount++
        }
    }
}

Do with it as you like. Again, in this case, there's no reason why you would use the tokenizer I think, unless you want to manually write all tags that aren't script tags to a separate buffer and process them some more. Just thought it worth mentioning here...


Random Number Generator from user touch input I have this in place for an RSA Key generation class. I force the user to swipe on the screen randomly and use that as a entropy pool.

class TouchRandom extends SecureRandom {

    private final Stack<Byte> pool = new Stack<>();

    @Override
    public synchronized void nextBytes(byte[] bytes) {
        for (int i = 0; i < bytes.length; i++) {
            bytes[i] = pool.pop();
        }
    }

    void addToPool(byte b) {
        pool.push(b);
    }

}

And here is how I feed the pool:

@Override
public boolean onTouchEvent(MotionEvent event) {
    touchRandom.addToPool((byte) event.getX());
    touchRandom.addToPool((byte) event.getY());
    return false;
}

Is this better or worse than a PRNG based on a Elliptic Curve or something else?

(Idea shamelessly stolen from VeraCrypt.)

This is a bad idea for different reasons. I'll short-list them and then go into details.

Your distribution isn't random. X and Y values are constrained to screensize, accordingly different devices have different possible randomness distributions, which is a "bad thing"™
User input is not random. "Guess a random number between 1 and 10" is not random and neither is "type on the screen randomly". Humans are really bad at faking randomness.

The x and y value constraints are a bad thing in the first place. Smaller devices have less entropy to generate, larger devices have repetition when the x and y coordinates "repeat", since you truncate any value larger than 8 bits.

Luckily screen-sizes are usually a nice multiple of \$256\times256\$ "tiles". The actually useful entropy you can currently extract accordingly is in one such tile.
Unfortunately this also means that "randomly" tapping similar areas in these tiles (as probably many non-technical people would do) will produce somewhat predictable bytes.

This directly brings us to the next point. The "Blue Seven Phenomenon". There's a metric ton of academic articles about the subject, but the bottom line is: Humans are bad at pretending to be random.
The brain is trained evolutionarily (and throughout our whole lives) to recognize patterns. True randomness doesn't actually exhibit those patterns. Any patterns we see in randomness are just ... imagination. Well not quite, but there's no actual meaning behind them.

The problem is when you ask users to provide "random" input, their brain will simply not be able to provide it, and instead you'll get a more or less pseudorandom pattern.

Long story short, you'll not receive something that I'd deem cryptographically secure.


Context manager for SuspendLayout I've found a few places where control updates take longer than the redraw cycle.

I started off surrounding them with Suspend and ResumeLayout:

container.SuspendLayout();
updateControls();
container.ResumeLayout(true);

After doing that a few times, I realized it could be refactored into a context:

/// <summary>
/// Context manager to suspend and resume layout engine while performing
/// updates that take longer than a refresh cycle
/// </summary>
/// <example>using (new SuspendLayoutContext(container)
///     DoSlowUpdate(container.Controls);</example>
class SuspendLayoutContext : IDisposable
{
    private readonly Control _control;
    private readonly bool _performLayoutAfter;

    /// <summary>
    /// Constructs the context manager to suspend and resume layout.
    /// </summary>
    /// <param name="container"></param>
    /// <param name="performLayout"></param>
    public SuspendLayoutContext(Control container, bool performLayoutAfter=true)
    {
        _control = container;
        _performLayoutAfter = performLayoutAfter;
        _control.SuspendLayout();
    }

    /// <summary>Calls ResumeLayout on the container.</summary>
    /// <remarks>If you call this manually, you are probably using this class wrong.</remarks>
    public void Dispose()
    {
        _control.ResumeLayout(_performLayoutAfter);
    }
}

Then, the above code would look like

using (new SuspendLayoutContext(container))
{
    updateControls();
}

I then decided I didn't like having to call the constructor every time, so I created an extension method (/// comments elided for brevity):

static class ControlExtensions
{
    public static SuspendLayoutContext SuspendLayoutContext(this Control container, bool performLayoutAfter=true)
    {
        return new SuspendLayoutContext(container, performLayoutAfter);
    }
}

The example then looks like

using (container.SuspendLayoutContext())
{
    updateControls();
}

Questions

Is there any reason not to use an IDisposable context here?  I'm more familiar with python, where I wouldn't hesitate to create new context managers - but the method name Dispose() raises red flags. (In python, it's __exit__.)
I debated between the extension method on Control and a static method on SuspendLayoutContext:

public static SuspendLayoutContext For(Control container, bool performLayoutAfter=true)
{
    return new SuspendLayoutContext(container, performLayoutAfter)
}

While I'm a big fan of fluent interfaces, the extension method felt cleaner.  What alternatives might be better?
I always miss something when I do this, what is it this time?

I think there is nothing wrong with using IDisposable whenever a class requires some sort of clean-up. It might have been originally designed to release unmanaged resources, but it is definitely no longer the case. Nowadays people use IDisposable to do all sort of things: unsubscribe from events, release pooled objects, flush buffers... this list goes on and on.

I also would prefer extension method over other options. It looks cleaner IMHO. What I don't like is the Context word. Not only because the method name should be a verb, but also because its somewhat misleading. I think a lot of people will assume that ...Context class is the class that implements context object (anti-)pattern. But your class doesn't. For example, class LayoutSuspender both sounds funny and describes what the class does. Win-win.

The thing you should be careful about is using this class with nested methods. For example, this:

public void UpdateAllControls()
{
    using (container.SuspendLayoutContext())
    {
        UpdateControlA();
        UpdateControlB();
    }
}

public void UpdateControlA()
{
    using (container.SuspendLayoutContext())
    {
        UpdateControlA();
    }
}

won't work as you want it to. So eventually you might want to implement some sort of reference counting.


Autofac support for connection strings I would like Autofac to instantiate components based on connection strings in the form of the URI:

autofac://assembly-name/full-class-name?param1=arg1&param2=arg2

So the following test will work:

[TestClass]
public class Locator_Should
{
    [TestMethod]
    public void Resolve()
    {
        var builder = new ContainerBuilder();
        builder.RegisterType<DiskFolder>();
        var container = builder.Build();
        var folder = container.Resolve<IFolder>(
            "autofac://LocatorTest/LocatorTest.DiskFolder?path=\\Windows");

        Assert.IsTrue(folder.Any());
    }
}

Where:

public interface IFolder : IEnumerable<string>
{
}

public class DiskFolder : ReadOnlyCollection<string>, IFolder
{
    public DiskFolder(string path)
        : base(Directory.GetFiles(path))
    {
    }
}

The library code is:

public static class AutofacLocator
{
    public static T Resolve<T>(this ILifetimeScope scope, string uri) =>
        scope.Resolve<T>(new Uri(uri));

    public static object Resolve(this ILifetimeScope scope, string uri) =>
        scope.Resolve(new Uri(uri));

    public static T Resolve<T>(this ILifetimeScope scope, Uri uri) =>
        (T)scope.Resolve(uri);

    public static object Resolve(this ILifetimeScope scope, Uri uri)
    {
        if (uri.Scheme != "autofac")
            throw new NotSupportedException("Schema not supported.");

        var factory = (Delegate)scope.Resolve(GetFactory(uri));
        return factory.DynamicInvoke(GetArguments(uri));
    }

    static Type GetFactory(Uri uri)
    {
        Func<Type[], Type> getType = Expression.GetFuncType;
        var types = GetParameters(uri)
            .Select(p => p.ParameterType)
            .Append(GetReturnType(uri));

        return getType(types.ToArray());
    }

    static object[] GetArguments(Uri uri)
    {
        var arguments = uri.Query
            .TrimStart('?')
            .Split('&')
            .Select(p => p.Split('='))
            .ToDictionary(nv => nv[0], nv => Uri.UnescapeDataString(nv[1]));

        return GetParameters(uri)
            .Select(p => Convert.ChangeType(arguments[p.Name], p.ParameterType))
            .ToArray();
    }

    static ParameterInfo[] GetParameters(Uri uri) =>
        GetReturnType(uri)
            .GetConstructors()
            .First()
            .GetParameters();

    static Type GetReturnType(Uri uri) =>
        Type.GetType($"{uri.Segments[1]}, {uri.Host}");
}

It appears the goal here is to provide parameters to a resolve operation using a different format, something string-based. This appears to be primarily useful in a service location (manually executing Resolve) scenario rather than true DI, so I would caution you to be careful.

Since this started as an Autofac solution, I'll post an answer based on Autofac.

My first recommendation would be to implement a new Parameter type rather than entirely separate handling of strings. This would make the solution more interoperable with other parameter types and Autofac on the whole. You can see how other parameter types like ResolvedParameter and NamedParameter are implemented to give you examples.

My second recommendation would be to look at how Autofac.Configuration handles parsing the string values into objects. Convert.ChangeType doesn't work on a lot of things and having a more robust mechanism will be valuable.

Finally... if possible, I might consider looking at the design that requires this and see if it can be changed. As I mentioned, this appears to be primarily useful in service location and it'd be better if you can switch your solution to use actual dependency injection. That'd mean the parameters need to be available either during registration time (e.g., read them from config and include their values when registering things); or during runtime, in which case you'd likely resolve a Func<Xy, Y, B> or something similar in your consuming class and not use a lifetime scope. In either case, the utility of this mechanism once you switch to DI is much more limited. It could be that just using Autofac.Configuration will suffice.


Detect the last resolve in an ES6 promise chain I searched but failed to find a canonical pattern for ES6 promise-based queue.  In fact all I am making is a step-wise FIFO buffer to ensure that messages don't collide so a very simple queue structure: like a message, sausage machine...

All it does is create a chain of promises, that resolve, at a fixed interval, to the messages in the order that they were received.

It's important for my application to know when the queue has been cleared and the only way I could think to do this was to add a Promise.all after the last message has gone into the queue (I use a setTimeout to simulate this).

To feed the Promise.all, I'm accumulating an array of references to the chained promises and just for the hell of it, shifting them out of the queue after they resolve.

I puzzled over how to do this without adding the array, like to somehow kick a .then(_ => console.log('done') ahead of each new promise. In other words a queue 'head' that I unplug from the previous last element and plug into the current added promise.  But I couldn't figure out how to achieve that.

Is there a better way than this?

'use strict';
var t = 500;

function write (x, enc, cb) {
  process.stdout.write('write: ' + x);
  cb(x-1)
}
function f(x, write) {
  return new Promise(res => {
    setTimeout(_ => write(x, null, res), t)
  })

}

console.log('start');

var q = Promise.resolve(0);
var qs = [q];

[1,2,3,4,5,6,7,8,9,10]
  .map(n =>  {
    return {
      cb: function (_) {
        console.log('\tcb ' + n);
        qs.shift();
        return _
      },
      x: n
    }
  })  // [{ cb, x }...]
  .forEach(
    (o) => {

      q = q.then(() =>
        f(o.x, write)   // return a promise to write
          .then(_ => o.cb(_ + 1)) // followed by a synchronous write
      );
      qs.push(q);

    });

// monitor queue
setTimeout(_ => {
  console.log('register all ' + qs.length);
  Promise.all(qs).then(a => {   // detect queue completed
    console.log('all done');
    a.forEach((_, i) => console.log(_ + '\t' + i))
  });
} , t*5);

Code in the question is hard to follow but it would appear that you are struggling to do something that's achieved very simply with this pattern, with which detection of "the last resolve in an ES6 promise chain" falls out in the wash. Read and understand that before continuing below.

If I'm correct, then you could write :

function makeAsyncSequence(arr, callback) {
    return arr.reduce((chain, item) => {
        return chain.then(() => callback(item));
    }, Promise.resolve());
}

And you would call :

makeAsyncSequence([1,2,3,4,5], doSomethingAsync).then(finalResult => {
    console.log('complete');
});

This is so simple that, in practice, we would normally choose to write arr.reduce(...) out longhand in our application everywhere it's needed. A special utility function like above is seldom needed.

Now, if you wish to impose a minimum delay that is not already imposed by the callback function :

First a delay function ...

function delay(t) {
    return new Promise(resolve => setTimeout(resolve, t));
};

... and an enhanced version of makeAsyncSequence(), in which callback(item) is aggregated with delay(minimumDelay) :

function makeAsyncSequence(arr, callback, minimumDelay) {
    minimumDelay = +minimumDelay || 0;
    return arr.reduce((chain, item) => {
        return chain.then(() => {
            if (minimumDelay) {
                return Promise.all([callback(item), delay(minimumDelay)]); // the slower of the two async processes determines the timing of the returned promise's success path.
            } else {
                return callback(item); // no minimumDelay required, therefore act as in the simple case.
            }
        });
    }, Promise.resolve());
}

And you would call :

makeAsyncSequence([1,2,3,4,5], doSomethingAsync, 1000).then(finalResult => {
    console.log('complete');
});

demo

Due to the increased complexity we would probably choose to write this as above in our application - ie as pair of special utilities.


A Simple Unix Filter in Racket - Learning the Racket Way I've written the following simple filter in Racket as my first Racket program and am wondering if I am writing it in an "idiomatic Racket style".

#! /usr/bin/env racket
#lang racket

(require racket/cmdline)

(define a-prolog-mode? (make-parameter #f))

;; parses the options passed on the command-line
(command-line
 #:program "patoms"
 #:once-any
 [("-a" "--aprolog") "output as A-Prolog code"
                     (a-prolog-mode? #t)])

;; dlv-input? : string -> boolean
;; Returns True if the given text corresponds to the output of DLV
;; and False otherwise. DLV's output is prefixed by one of the following
;; strings: "DLV", "{", or "Best model".
(define (dlv-input? text)
  (regexp-match? #rx"^DLV|^{|^Best model" text))

;; text->answer-sets : string -> list of strings
;; Returns a list comprised of all of the answer sets in the given text.
(define (text->answer-sets text)
  (cond
    [(dlv-input? text) (regexp-match* #rx"{(.*?)}" text #:match-select cadr)]
    [else null]))

;; write-as-code : string
;; Writes the given answer set in plain text form to standard output.
(define (write-as-text answer-set)
  (let ([literals (map string-trim (string-split answer-set ","))])
    (cond
      [(empty? literals) (printf "~a~n" "{}")]
      [else (for-each (λ (literal) (printf "~a~n" literal)) literals)])
    (printf "~a~n" "::endmodel")))

;; write-as-code : string
;; Writes the given answer set as A-Prolog code to standard output.
(define (write-as-code answer-set)
  (let ([literals (map string-trim (string-split answer-set ","))])
    (for-each (λ (literal) (printf "~a.~n" literal)) literals)
    (printf "~a~n" "%%endmodel")))

;; main
;; Serves as the main function of the program. If a-prolog-mode is specified
;; by the user via the command-line, all of the answer sets that may be parsed
;; from standard input are written to standard output as A-Prolog. Otherwise
;; the answer sets are written in a plain text format.
(define (main)
  (let* ([text (port->string (current-input-port))]
         [answer-sets (text->answer-sets text)])
    (cond
      [(a-prolog-mode?) (for-each write-as-code answer-sets)]
      [else (for-each write-as-code answer-sets)])))

(main)

In general, any and all feedback would be very much appreciated given that this is my first foray into Racket.

Thank you kindly in advance.

Overall, this is very pleasant code to review.  Especially given you're new to the language.  Your function names are follow convention such as the use of ending predicates with ? and indicating conversions with ->.  The comments make this code easy to understand.  So bravo, keep at it!  That being said, here are some pretty minor suggestions.

"Favor define when feasible".  Your let's and let*'s could be changed to internal define's to decrease nesting.
Change (define (main) ...) to (module+ main ...).  Submodule support added in June's 5.3 release make having main's and test's easier than ever.
Within main, should one of the cond branches be using write-as-text instead of write-as-code?  right now, both branches do the same thing.  Along the same vein, the comment above write-as-text was copied but not changed from ;; write-as-code.
In racket, we have 3 ways of doing output, display, write, and print.  Since your write-as-text/code functions use printing, I would change the names of those functions to be called print-as-text/code


Function to extract bytes as a decimal You give this function a pointer and how many bytes you want to extract as a decimal and it returns it.

long long
byte_extract
(void * source, size_t bytes)
{
    long long num = 0, m = 1;
    size_t i;

    for(i = 0; i != bytes; i++)
    {
        num += (((unsigned char*)source)[i]*m);
        m<<=0x8;
    }
    return num;
}

It does the job, so that's good. It's quite clear and I don't see fundamental flaws in it. There are a few minor things I'd recommend to change:

The most important expression in the middle is a bit hard to read.
I find it much easier this way: num += m * ((unsigned char*) source)[i];
(Redundant parens removed, multiplication operands switched, minor spacing adjustments)
Instead of m<<=0x8; I would write m <<= 8; with more spacing and 0x8 is the same as simply 8
The name byte_extract doesn't convey well what you described: "You give this function a pointer and how many bytes you want to extract as a decimal and it returns it." extract_num or extract_bytes_as_num would be better
Renaming m to multiplier could be a good idea too


Make JSON based on available fields I am trying to create a filter string according to the fields available with the following code, how could i improve the following with a switch case or better way?

   if ((this.selectedCity === 'City') && this.selectedCategory === 'Category' && this.fromDate === '') {
            this.filter = `{}`;
        } else if ((this.selectedCity !== 'City') && this.selectedCategory === 'Category' && this.fromDate === '') {
            this.filter = `{"location.city":"${this.selectedCity}"}`;
        } else if ((this.selectedCity === 'City') && this.selectedCategory !== 'Category' && this.fromDate === '') {
            this.filter = `{"category":"${this.selectedCategory}"}`;
        } else if ((this.selectedCity === 'City') && this.selectedCategory !== 'Category' && this.fromDate !== '') {
            this.filter = `{"date.start":{"$gte":"${this.fromDate}"},"date.end":{"$lt":"${this.toDate}"}}`;
        } else if ((this.selectedCity !== 'City') && this.selectedCategory !== 'Category' && this.fromDate === '') {
            this.filter = `{"location.city":"${this.selectedCity}","category":"${this.selectedCategory}"}`;
        } else if ((this.selectedCity !== 'City') && this.selectedCategory === 'Category' && this.fromDate !== '') {
            this.filter = `{"location.city":"${this.selectedCity}","date.start":{"$gte":"${this.fromDate}"},"date.end":{"$lt":"${this.toDate}"}}`;
        } else if ((this.selectedCity === 'City') && this.selectedCategory === 'Category' && this.fromDate !== '') {
            this.filter = `{"date.start":{"$gte":"${this.fromDate}"},"date.end":{"$lt":"${this.toDate}"}}`;
        } else if ((this.selectedCity !== 'City') && this.selectedCategory !== 'Category' && this.fromDate !== '') {
            this.filter = `{"location.city":"${this.selectedCity}","category":"${this.selectedCategory}","date.start":{"$gte":"${this.fromDate}"},"date.end":{"$lt":"${this.toDate}"}}`;
        }

I recommend building the filter in steps, starting with an empty object {}. You then add attributes according to the selected city, category and date. And finally, you convert it to a JSON string via JSON.stringify(filter):

let filter = {};

if (this.selectedCity !== 'City') {
  filter.location = {
    city: this.selectedCity
  };
}

if (this.selectedCategory !== 'Category') {
  filter.category = this.selectedCategory;
}

if (this.fromDate !== '') {
  filter.date = {
    start: {'$gte': this.fromDate},
    end:   {'$lt':  this.toDate}
  };
}

this.filter = JSON.stringify(filter);


Producer - multiple consumer implementation using futures I've tried to implement a producer-multiple consumer pattern using futures and I was wondering should it ever deadlock, and if it should, why?

I have this method RunNTimes which I implemented in order to run the implementation, e.g. 1000 times with 500 consumers.

The implementation shouldn't deadlock as far as I know and it often doesn't (running it multiple times and telling RunNTimes to repeat for 1000 times). However, every once in a while it blocks at the first or second run (I'm running it from sbt). For the sake of simplicity, ignore the RunNTimes method, I've included it only to explain why the occasional deadlock seems strange to me.

package comparison_examples.futuresimplementation

import common.MeasurementHelpers._
import common._

import scala.collection.mutable
import scala.concurrent.blocking
import scala.concurrent.ExecutionContext.Implicits.global
import scala.concurrent.duration.Duration
import scala.concurrent.{Await, Future, Promise}

case class Item(value: Int)

object ProducerConsumer {

    private val sharedQueue = mutable.Queue[Item]()

    def main(args: Array[String]): Unit = {

        val n: Double = if (args.nonEmpty) args(0).toDouble else Configuration.runTimes.toDouble
        val numConsumers = if (args.length == 2) args(1).toInt else Configuration.numberOfConsumers

        val funRunNTimes = MeasurementHelpers.runNTimes(n.toInt) _

        val results = funRunNTimes {

            val p = Promise[Boolean]()

            val producer = new Producer(p, sharedQueue)
            val cs = startConsumers(numConsumers, List[Consumer](), sharedQueue, p)

            val fp = producer.start()
            val fs = cs.map(x => x.start())

            val f = Future.sequence(fs)

            Await.result(fp, Duration.Inf)
            Await.result(f, Duration.Inf)

            val allElementsObtained = cs.flatMap(_.getObtainedItems)

            println(allElementsObtained.length)
        }

        println(s"Run times: $n")
        println(s"Average number of distinct threads: ${results.map(x => x._3.length).sum / n }")
        println(s"Average duration: ${results.map(x => x._2).sum / n} milliseconds")

        println()
    }

    def startConsumers(max: Int, result: List[Consumer], sharedQueue: mutable.Queue[Item], p: Promise[Boolean]): List[Consumer] =
         if (max == 0) result
         else {
             val consumer = new Consumer(p, sharedQueue)

             startConsumers(max - 1, consumer :: result, sharedQueue, p)
        }
 }

class Producer(p: Promise[Boolean], sharedQueue: mutable.Queue[Item]) {

def start(): Future[Unit] = Future {

    val n = Configuration.workToProduce

        for (i <- 1 to n) {

            val item = Item(i)

                sharedQueue.synchronized {

                    sharedQueue.enqueue(item)
                    sharedQueue.notifyAll()
                }
        }

    p success true
}
}

class Consumer(p: Promise[Boolean], sharedQueue: mutable.Queue[Item]) {

private var obtainedItems = List[Item]()

def getObtainedItems: List[Item] = obtainedItems

def start(): Future[Unit] = Future {

    addCurrentThread()

        while (sharedQueue.nonEmpty || !p.isCompleted) {

            obtainedItems = getItem match {

                case None => obtainedItems
                case Some(item) => item :: obtainedItems
            }
        }
}

def getItem: Option[Item] = blocking {

    sharedQueue.synchronized {

        while (sharedQueue.isEmpty && !p.isCompleted) {

            sharedQueue.wait()
        }

        val result = if (sharedQueue.nonEmpty)
            Some(sharedQueue.dequeue())
        else
            None

        sharedQueue.notifyAll()

        result
    }
}

def printObtainedItems(): Unit =
    obtainedItems.foreach(x => print(s"\t ${x.value}"))
}

I notice several things.

You have a data race on the shared queue. The consumer has an unsynchronized access to the nonEmpty property.

The notifyAll after you removed an element from the queue is unnecessary. Consumers shouldn't care about other consumers removing items.

But the main problem, which is suspect is the source of your deadlocks, is the race condition between the producer and the consumers. Here's what I think happens:

The producer produces the last item. It notifies all threads, releases the lock, but is preempted before setting the promise.
Consumer C1 wakes up, sees the queue is non-empty and grabs the element. It notifies all threads, releases the lock, and goes out to add the element to its list.
The other consumers wake up, see that the queue is empty but the promise is not set, and go back to sleep.
The producer sets the promise.
Consumer C1 sees that the promise is set and terminates.
The other consumers continue sleeping for all eternity.

To solve this, you need to notifyAll after you've set the promise. You can also probably optimize the whole thing by only calling notifyOne after adding an element.


A function that finds the kth smallest integer in array A (unsorted), of size n int kth_smallest_number(int A[], int n, int k){
    /*kth smallest number is smaller than pr equal to n-k numbers in A*/
    int smaller_than_count;
    int i,j;
    for(i=0; i<n; i++){
        /*Check whether the number A[i] satisfies
        the condition*/
        smaller_than_count = 0;
        for(j=0; j<n; j++){
            if(A[i]<=A[j]){
                smaller_than_count++;
            }
        }
        if(smaller_than_count == n-k){
            return A[i];
        }
    }
}

I just wanted to know if this solution is right, I am not worried about efficiency right now just correctness.

The compiler is your friend

Activate all warnings and you'll get pretty good hints :

selection.c: In function ‘kth_smallest_number’:
selection.c:17:1: warning: control reaches end of non-void function [-Wreturn-type]
 }
 ^

Test your code

Here's what I have written to test your code :

int A[] = {4,4,56,78,54,6512,21,8,7,2,321,87,78,54,2,21,5,4,321,456,5,4,35,5,4,1,67,8,1,54,4,654,7,6,54,321,7,32,4,21,54,7};
int n = sizeof(A)/sizeof(A[0]);
for (int k=0; k<n; k++)
{
    printf("%d %d\n",k,kth_smallest_number(A,n,k));
}

I'd expect the returned value to be sorted and it's clearly not the case here. Thus, I expect something to be wrong.

Playing with other inputs : int A[] = {10,9,8,7,6,5,4,3,2,1,0}; , int A[] = {10,9,8,7,6,5,5,4,3,2,1,0};, it seems like your solution does not handle properly duplicated value.

A matter of style

The indentation looks weird on your question.

You should try to declare your local variable in the smallest possible scope. Among other things, you can define your loop indices in the loop syntax in you use the C99 mode (-std=c99).

Once rewritten (but still wrong), your code looks like :

int kth_smallest_number(int A[], int n, int k){
    for(int i=0; i<n; i++){
        /*Check whether the number A[i] satisfies
          the condition*/
        int smaller_than_count = 0;
        for(int j=0; j<n; j++){
            if(A[i]<=A[j]){
                smaller_than_count++;
            }
        }
        if(smaller_than_count == n-k){
            return A[i];
        }
    }
}

Edit :

An attempt to correctness

Here what I have written , it seems to work fine but I won't promise anything.

int kth_smallest_number(int A[], int n, int k){
    for(int i=0; i<n; i++){
        /*Check whether the number A[i] satisfies the condition*/
        int ai = A[i];
        int smaller_than_count = 0;
        int bigger_than_count = 0;
        for(int j=0; j<n; j++){
            int aj = A[j];
            if (ai < aj) {
                smaller_than_count++;
                if (smaller_than_count >= n-k) break; // Optimisation
            } else if (ai > aj) {
                bigger_than_count++;
                if (bigger_than_count > k) break; // Optimisation
            }
        }
        if(smaller_than_count < n-k && bigger_than_count <= k){
            return ai;
        }
    }
    return -1;
}


Evaluation of insert method from within a class using PDO I am learning PDO. I have a class -  User...

class User {
    protected static $table_name="users";
    public $id;
    public $first_name;
    public $last_name;

    public function pdo_create() {
        global $handler;
        $sql = " INSERT INTO users (first_name, last_name)";
        $sql .= " VALUES (?, ?)";
        $query = $handler->prepare($sql);;
        $query->execute(array($this->first_name ,$this->last_name));
    }
}

When I run this...

$user= new User();
$user->first_name = "test";
$user->last_name = "test";
$user->pdo_create();

It works fine. I know it's a bit odd posting when things are working, but I have suspicions that this is not the best approach. I am still learning PDO and about the convert my MySQLi site over to PDO. I want to make sure the PDO veterans think this is a reasonable approach.

Your design is flawed in the following way:

DO NOT USE GLOBAL.

It is not OO programming and you shouldn't be calling it if you are using PDO - pass your PDO through your function or as part of your user constructors. (see SOLID design patterns)

class User
{
     //private attributes
     private $_db;
     public function __construct(PDO $pdo)
     {
         $this->_db = $pdo;
     }
}

your function pdo_create
is too couple with pdo - if you decide to switch off from pdo - you will need to re-write that function call to reflect what you change to - use convention a bit more general: creatingUser(PDO $pdo) for example or even better creatingUser(DBInterface $db) (as part of SOLID

Now as for the function itself:

+1 for using prepare statement however ? is more used for mysqli - it better to use place handler instead. such as VALUES(:first_name, :last_name). When you pass the array you can then use the place handler to effectively assign to which variable its value. It is best for readability later.

Finally: you defeat the purpose of the class if your attributes are PUBLIC. Ideally functions receives parameters by the function itself OR by the constructor of the class which initializes the attributes of the class.


Is this common way to build this class and method? I am a new developer therefore I have many questions:

In this class I get file path on the disk (wireshark file that needs to be converted to pcap format extension) and convert it with the method convertFileToPcap(), in the constructor I am calling convertFileToPcap() and from the main after create the Editcap object I am receiving the new file path (pcap format) with the property getNewFileName() who return _newFileName (class member) and I want to know if there is a better way or appropriate way to do it.

public class Editcap
{
    #region members
    private string _editpcap;
    private ProcessStartInfo _editpcapProcess;
    private FileInfo _fileInfo;
    private string _newFileName;
    #endregion

    #region c'tor
    public Editcap(FileInfo fileinfo)
    {
        _fileInfo = fileinfo;
        _newFileName = "";
        convertFileToPcap();
    }
    #endregion

    public void convertFileToPcap()
    {
        string oldFileExtension = _fileInfo.Extension;
        _newFileName = _fileInfo.FullName.Replace(oldFileExtension, "_new") + ".pcap";
        _editpcapProcess = new ProcessStartInfo(string.Format("\"{0}\"", _editpcap));
        _editpcapProcess.Arguments = (string.Format("{2}{0}{2} -F libpcap {2}{1}{2}", _fileInfo.FullName, _newFileName, "\""));
        _editpcapProcess.WindowStyle = ProcessWindowStyle.Hidden;
        _editpcapProcess.RedirectStandardOutput = true;
        _editpcapProcess.RedirectStandardError = true;
        _editpcapProcess.CreateNoWindow = true;
        _editpcapProcess.UseShellExecute = false;
        _editpcapProcess.ErrorDialog = false;
        Process capinfosProcess = Process.Start(_editpcapProcess);
        capinfosProcess.WaitForExit();
    }

    public string getNewFileName()
    {
        return _newFileName;
    }
}

Hello and welcome to Code Review!

I'd like to first direct you to Microsoft's official naming
guidelines for C# development. Your code goes against a few of
those, and if that's your company's or team's guidelines, than great!
If you don't have an official style, start with the ones I linked to.
I'd definitely not call the convert method from the constructor.
This could throw exceptions that would prevent the object from being
created. Keep your processing methods separate and let the caller
call it. It IS public, after all!
your member variables seem a little too, well, global, in some cases.
Only _fileInfo and _newFileName are used in more than one method.
Keep the others local to their methods.

As part of this, note that _editpcap never gets assigned and is
always null. I don't think that's what you're intending.

the getNewFileName method looks very Java-like as Java didn't have
the concept of properties. This is a prime candidate for being made a
property.
the Process class is IDisposable, so wrapping it in a using
block is idiomatic.
now I'm just getting into minor nitpicking, the fields set in the
constructor are invariant after construction, so make the intent
known by using the readonly keyword. Also, you can employ modern C#
amenities as object initializers and the var keyword.

All this being said, here is my proposed refactor of the code:

public class Editcap
{
    #region members
    private readonly FileInfo fileInfo;
    private string newFileName = string.Empty;
    #endregion

    #region c'tor
    public Editcap(FileInfo fileinfo)
    {
        if (fileinfo == null)
        {
            throw new ArgumentNullException("fileInfo");
        }

        this.fileInfo = fileinfo;
    }
    #endregion

    public void ConvertFileToPcap()
    {
        this.newFileName = this.fileInfo.FullName.Replace(this.fileInfo.Extension, "_new") + ".pcap";

        string editpcap = null; // still not set, need to fix!
        var editpcapProcess = new ProcessStartInfo(string.Format("\"{0}\"", editpcap))
        {
            Arguments = string.Format("{2}{0}{2} -F libpcap {2}{1}{2}", this.fileInfo.FullName, this.newFileName, "\""),
            WindowStyle = ProcessWindowStyle.Hidden,
            RedirectStandardOutput = true,
            RedirectStandardError = true,
            CreateNoWindow = true,
            UseShellExecute = false,
            ErrorDialog = false
        };

        using (var capinfosProcess = Process.Start(editpcapProcess))
        {
            capinfosProcess.WaitForExit();
        }
    }

    public string NewFileName
    {
        get
        {
            return this.newFileName;
        }
    }
}


Diffie-Hellman in JavaScript Diffie-Hellman is a key exchange that allows 2 people to share a symmetric key without interaction beforehand. First, a person shares an equation; in this case, we use:

$$3^x \mod{17}$$

Next, each person generates a random, usually prime, number. Then, they plug it in the equation. Let's use 5 and 7:

Alice: \$3^5 \mod{17} \equiv 9 \$
Bob: \$3^7 \mod{17} \equiv 6 \$

Finally, they share their generated numbers and calculate a "Shared Secret", this is done by (\$U\$ represents your private number):

\$\text{Public}^U \mod{17}\$

In this case:

Alice: \$6^5 \mod{17} \equiv 2\$
Bob: \$9^7 \mod{17} \equiv 2\$

Resulting in both parties having the same number. Confused? See this video.

I've implemented a function to generate that key. Thoughts?

function KeyGen(p1, p2, n){
  // Get a binary string, reverse it
  var bin = String((n).toString(2)).split("").reverse().join("");
  // Base for growth
  var grow = n;
  // Holds values for totals
  var tota = [];
  var total = 1;
  // The main loop
  for(var i = 0; i < bin.length; i++){
    tota[i] = 1;
    if(bin.substring(i, i + 1) === "1"){
      for (var l = grow; l > 0;l--){
        tota[i] *= p1;
        tota[i] %= p2;
      }
    }
    total *= tota[i];
    total %= p2;
    grow *= n;
  }
  return total;
}

// Example of gen
var gen = KeyGen(3, 17, 5);

Variable Naming

It took me a while to figure out what are p1, p2 and n (and you don't mention them in your example).

String operation

You could use math operation to do the computation:

function KeyGen(base, modulo, exponent) {
  var result = 1;
  while(exponent > 0){
    if(exponent % 2 == 1){
        result = (result * base) % modulo;
    }
    base = (base * base) % modulo;
    exponent = exponent >>> 1;
  }
  return result;
}

The memory footprint is then very small (no array required).

Big numbers

Issues appear when you try to use big integers (more than 54 bits - like in your comment).
To handle them correctly, I would recommand an external library like BigInt:

var bigInt=function(e){"use strict";function o(e,t){this.value=e,this.sign=t,this.isSmall=!1}function u(e){this.value=e,this.sign=e<0,this.isSmall=!0}function a(e){return-r<e&&e<r}function f(e){return e<1e7?[e]:e<1e14?[e%1e7,Math.floor(e/1e7)]:[e%1e7,Math.floor(e/1e7)%1e7,Math.floor(e/1e14)]}function l(e){c(e);var n=e.length;if(n<4&&O(e,i)<0)switch(n){case 0:return 0;case 1:return e[0];case 2:return e[0]+e[1]*t;default:return e[0]+(e[1]+e[2]*t)*t}return e}function c(e){var t=e.length;while(e[--t]===0);e.length=t+1}function h(e){var t=new Array(e),n=-1;while(++n<e)t[n]=0;return t}function p(e){return e>0?Math.floor(e):Math.ceil(e)}function d(e,n){var r=e.length,i=n.length,s=new Array(r),o=0,u=t,a,f;for(f=0;f=u?1:0,s[f]=a-o*u;while(f<r)a=e[f]+o,o=a===u?1:0,s[f++]=a-o*u;return o>0&&s.push(o),s}function v(e,t){return e.length>=t.length?d(e,t):d(t,e)}function m(e,n){var r=e.length,i=new Array(r),s=t,o,u;for(u=0;u<r;u++)o=e[u]-s+n,n=Math.floor(o/s),i[u]=o-n*s,n+=1;while(n>0)i[u++]=n%s,n=Math.floor(n/s);return i}function g(e,n){var r=e.length,i=n.length,s=new Array(r),o=0,u=t,a,f;for(a=0;a<i;a++)f=e[a]-o-n[a],f<0?(f+=u,o=1):o=0,s[a]=f;for(a=i;a<r;a++){f=e[a]-o;if(!(f<0)){s[a++]=f;break}f+=u,s[a]=f}for(;a<r;a++)s[a]=e[a];return c(s),s}function y(e,t,n){var r,i;return O(e,t)>=0?r=g(e,t):(r=g(t,e),n=!n),r=l(r),typeof r=="number"?(n&&(r=-r),new u(r)):new o(r,n)}function b(e,n,r){var i=e.length,s=new Array(i),a=-n,f=t,c,h;for(c=0;c<i;c++)h=e[c]+a,a=Math.floor(h/f),h%=f,s[c]=h<0?h+f:h;return s=l(s),typeof s=="number"?(r&&(s=-s),new u(s)):new o(s,r)}function w(e,n){var r=e.length,i=n.length,s=r+i,o=h(s),u=t,a,f,l,p,d;for(l=0;l<r;++l){p=e[l];for(var v=0;v<i;++v)d=n[v],a=p*d+o[l+v],f=Math.floor(a/u),o[l+v]=a-f*u,o[l+v+1]+=f}return c(o),o}function E(e,n){var r=e.length,i=new Array(r),s=t,o=0,u,a;for(a=0;a<r;a++)u=e[a]*n+o,o=Math.floor(u/s),i[a]=u-o*s;while(o>0)i[a++]=o%s,o=Math.floor(o/s);return i}function S(e,t){var n=[];while(t-->0)n.push(0);return n.concat(e)}function x(e,t){var n=Math.max(e.length,t.length);if(n<=400)return w(e,t);n=Math.ceil(n/2);var r=e.slice(n),i=e.slice(0,n),s=t.slice(n),o=t.slice(0,n),u=x(i,o),a=x(r,s),f=x(v(i,r),v(o,s));return v(v(u,S(g(g(f,u),a),n)),S(a,2*n))}function T(e,n,r){return e<t?new o(E(n,e),r):new o(w(n,f(e)),r)}function N(e){var n=e.length,r=h(n+n),i=t,s,o,u,a,f;for(u=0;u<n;u++){a=e[u];for(var l=0;l<n;l++)f=e[l],s=a*f+r[u+l],o=Math.floor(s/i),r[u+l]=s-o*i,r[u+l+1]+=o}return c(r),r}function C(e,n){var r=e.length,i=n.length,s=t,o=h(n.length),u=n[i-1],a=Math.ceil(s/(2*u)),f=E(e,a),c=E(n,a),p,d,v,m,g,y,b;f.length<=r&&f.push(0),c.push(0),u=c[i-1];for(d=r-i;d>=0;d--){p=s-1,f[d+i]!==u&&(p=Math.floor((f[d+i]*s+f[d+i-1])/u)),v=0,m=0,y=c.length;for(g=0;g<y;g++)v+=p*c[g],b=Math.floor(v/s),m+=f[d+g]-(v-b*s),v=b,m<0?(f[d+g]=m+s,m=-1):(f[d+g]=m,m=0);while(m!==0){p-=1,v=0;for(g=0;g<y;g++)v+=f[d+g]-s+c[g],v<0?(f[d+g]=v+s,v=0):(f[d+g]=v,v=1);m+=v}o[d]=p}return f=L(f,a)[0],[l(o),l(f)]}function k(e,n){var r=e.length,i=n.length,s=[],o=[],u=t,a,f,c,h,p;while(r){o.unshift(e[--r]);if(O(o,n)<0){s.push(0);continue}f=o.length,c=o[f-1]*u+o[f-2],h=n[i-1]*u+n[i-2],f>i&&(c=(c+1)*u),a=Math.ceil(c/h);do{p=E(n,a);if(O(p,o)<=0)break;a--}while(a);s.push(a),o=g(o,p)}return s.reverse(),[l(s),l(o)]}function L(e,n){var r=e.length,i=h(r),s=t,o,u,a,f;a=0;for(o=r-1;o>=0;--o)f=a*s+e[o],u=p(f/n),a=f-u*n,i[o]=u|0;return[i,a|0]}function A(e,n){var r,i=Q(n),s=e.value,a=i.value,c;if(a===0)throw new Error("Cannot divide by zero");if(e.isSmall)return i.isSmall?[new u(p(s/a)),new u(s%a)]:[G[0],e];if(i.isSmall){if(a===1)return[e,G[0]];if(a==-1)return[e.negate(),G[0]];var h=Math.abs(a);if(h<t){r=L(s,h),c=l(r[0]);var d=r[1];return e.sign&&(d=-d),typeof c=="number"?(e.sign!==i.sign&&(c=-c),[new u(c),new u(d)]):[new o(c,e.sign!==i.sign),new u(d)]}a=f(h)}var v=O(s,a);if(v===-1)return[G[0],e];if(v===0)return[G[e.sign===i.sign?1:-1],G[0]];s.length+a.length<=200?r=C(s,a):r=k(s,a),c=r[0];var m=e.sign!==i.sign,g=r[1],y=e.sign;return typeof c=="number"?(m&&(c=-c),c=new u(c)):c=new o(c,m),typeof g=="number"?(y&&(g=-g),g=new u(g)):g=new o(g,y),[c,g]}function O(e,t){if(e.length!==t.length)return e.length>t.length?1:-1;for(var n=e.length-1;n>=0;n--)if(e[n]!==t[n])return e[n]>t[n]?1:-1;return 0}function M(e){var t=e.abs();if(t.isUnit())return!1;if(t.equals(2)||t.equals(3)||t.equals(5))return!0;if(t.isEven()||t.isDivisibleBy(3)||t.isDivisibleBy(5))return!1;if(t.lesser(25))return!0}function H(e){return(typeof e=="number"||typeof e=="string")&&+Math.abs(e)<=t||e instanceof o&&e.value.length<=1}function B(e,t,n){t=Q(t);var r=e.isNegative(),i=t.isNegative(),s=r?e.not():e,o=i?t.not():t,u=[],a=[],f=!1,l=!1;while(!f||!l)s.isZero()?(f=!0,u.push(r?1:0)):r?u.push(s.isEven()?1:0):u.push(s.isEven()?0:1),o.isZero()?(l=!0,a.push(i?1:0)):i?a.push(o.isEven()?1:0):a.push(o.isEven()?0:1),s=s.over(2),o=o.over(2);var c=[];for(var h=0;h<u.length;h++)c.push(n(u[h],a[h]));var p=bigInt(c.pop()).negate().times(bigInt(2).pow(c.length));while(c.length)p=p.add(bigInt(c.pop()).times(bigInt(2).pow(c.length)));return p}function I(e){var n=e.value,r=typeof n=="number"?n|j:n[0]+n[1]*t|F;return r&-r}function q(e,t){return e=Q(e),t=Q(t),e.greater(t)?e:t}function R(e,t){return e=Q(e),t=Q(t),e.lesser(t)?e:t}function U(e,t){e=Q(e).abs(),t=Q(t).abs();if(e.equals(t))return e;if(e.isZero())return t;if(t.isZero())return e;var n=G[1],r,i;while(e.isEven()&&t.isEven())r=Math.min(I(e),I(t)),e=e.divide(r),t=t.divide(r),n=n.multiply(r);while(e.isEven())e=e.divide(I(e));do{while(t.isEven())t=t.divide(I(t));e.greater(t)&&(i=t,t=e,e=i),t=t.subtract(e)}while(!t.isZero());return n.isUnit()?e:e.multiply(n)}function z(e,t){return e=Q(e).abs(),t=Q(t).abs(),e.divide(U(e,t)).multiply(t)}function W(e,n){e=Q(e),n=Q(n);var r=R(e,n),i=q(e,n),s=i.subtract(r);if(s.isSmall)return r.add(Math.round(Math.random()*s));var a=s.value.length-1,f=[],c=!0;for(var h=a;h>=0;h--){var d=c?s.value[h]:t,v=p(Math.random()*d);f.unshift(v),v<d&&(c=!1)}return f=l(f),r.add(typeof f=="number"?new u(f):new o(f,!1))}function V(e){var t=e.value;return typeof t=="number"&&(t=[t]),t.length===1&&t[0]<=36?"0123456789abcdefghijklmnopqrstuvwxyz".charAt(t[0]):"<"+t+">"}function $(e,t){t=bigInt(t);if(t.isZero()){if(e.isZero())return"0";throw new Error("Cannot convert nonzero numbers to base 0.")}if(t.equals(-1))return e.isZero()?"0":e.isNegative()?(new Array(1-e)).join("10"):"1"+(new Array(+e)).join("01");var n="";e.isNegative()&&t.isPositive()&&(n="-",e=e.abs());if(t.equals(1))return e.isZero()?"0":n+(new Array(+e+1)).join(1);var r=[],i=e,s;while(i.isNegative()||i.compareAbs(t)>=0){s=i.divmod(t),i=s.quotient;var o=s.remainder;o.isNegative()&&(o=t.minus(o).abs(),i=i.next()),r.push(V(o))}return r.push(V(i)),n+r.reverse().join("")}function J(e){if(a(+e)){var t=+e;if(t===p(t))return new u(t);throw"Invalid integer: "+e}var r=e[0]==="-";r&&(e=e.slice(1));var i=e.split(/e/i);if(i.length>2)throw new Error("Invalid integer: "+f.join("e"));if(i.length===2){var s=i[1];s[0]==="+"&&(s=s.slice(1)),s=+s;if(s!==p(s)||!a(s))throw new Error("Invalid integer: "+s+" is not a valid exponent.");var f=i[0],l=f.indexOf(".");l>=0&&(s-=f.length-l,f=f.slice(0,l)+f.slice(l+1));if(s<0)throw new Error("Cannot include negative exponent part for integers");f+=(new Array(s+1)).join("0"),e=f}var h=/^([0-9][0-9]*)$/.test(e);if(!h)throw new Error("Invalid integer: "+e);var d=[],v=e.length,m=n,g=v-m;while(v>0)d.push(+e.slice(g,v)),g-=m,g<0&&(g=0),v-=m;return c(d),new o(d,r)}function K(e){return a(e)?new u(e):J(e.toString())}function Q(e){return typeof e=="number"?K(e):typeof e=="string"?J(e):e}var t=1e7,n=7,r=9007199254740992,i=f(r),s=Math.log(r);o.prototype.add=function(e){var t,n=Q(e);if(this.sign!==n.sign)return this.subtract(n.negate());var r=this.value,i=n.value;return n.isSmall?new o(m(r,Math.abs(i)),this.sign):new o(v(r,i),this.sign)},o.prototype.plus=o.prototype.add,u.prototype.add=function(e){var t=Q(e),n=this.value;if(n<0!==t.sign)return this.subtract(t.negate());var r=t.value;if(t.isSmall){if(a(n+r))return new u(n+r);r=f(Math.abs(r))}return new o(m(r,Math.abs(n)),n<0)},u.prototype.plus=u.prototype.add,o.prototype.subtract=function(e){var t=Q(e);if(this.sign!==t.sign)return this.add(t.negate());var n=this.value,r=t.value;return t.isSmall?b(n,Math.abs(r),this.sign):y(n,r,this.sign)},o.prototype.minus=o.prototype.subtract,u.prototype.subtract=function(e){var t=Q(e),n=this.value;if(n<0!==t.sign)return this.add(t.negate());var r=t.value;return t.isSmall?new u(n-r):b(r,Math.abs(n),n>=0)},u.prototype.minus=u.prototype.subtract,o.prototype.negate=function(){return new o(this.value,!this.sign)},u.prototype.negate=function(){var e=this.sign,t=new u(-this.value);return t.sign=!e,t},o.prototype.abs=function(){return new o(this.value,!1)},u.prototype.abs=function(){return new u(Math.abs(this.value))},o.prototype.multiply=function(e){var n,r=Q(e),i=this.value,s=r.value,u=this.sign!==r.sign,a;if(r.isSmall){if(s===0)return G[0];if(s===1)return this;if(s===-1)return this.negate();a=Math.abs(s);if(a<t)return new o(E(i,a),u);s=f(a)}return i.length+s.length>4e3?new o(x(i,s),u):new o(w(i,s),u)},o.prototype.times=o.prototype.multiply,u.prototype._multiplyBySmall=function(e){return a(e.value*this.value)?new u(e.value*this.value):T(Math.abs(e.value),f(Math.abs(this.value)),this.sign!==e.sign)},o.prototype._multiplyBySmall=function(e){return e.value===0?G[0]:e.value===1?this:e.value===-1?this.negate():T(Math.abs(e.value),this.value,this.sign!==e.sign)},u.prototype.multiply=function(e){return Q(e)._multiplyBySmall(this)},u.prototype.times=u.prototype.multiply,o.prototype.square=function(){return new o(N(this.value),!1)},u.prototype.square=function(){var e=this.value*this.value;return a(e)?new u(e):new o(N(f(Math.abs(this.value))),!1)},o.prototype.divmod=function(e){var t=A(this,e);return{quotient:t[0],remainder:t[1]}},u.prototype.divmod=o.prototype.divmod,o.prototype.divide=function(e){return A(this,e)[0]},u.prototype.over=u.prototype.divide=o.prototype.over=o.prototype.divide,o.prototype.mod=function(e){return A(this,e)[1]},u.prototype.remainder=u.prototype.mod=o.prototype.remainder=o.prototype.mod,o.prototype.pow=function(e){var t=Q(e),n=this.value,r=t.value,i,s,o;if(r===0)return G[1];if(n===0)return G[0];if(n===1)return G[1];if(n===-1)return t.isEven()?G[1]:G[-1];if(t.sign)return G[0];if(!t.isSmall)throw new Error("The exponent "+t.toString()+" is too large.");if(this.isSmall&&a(i=Math.pow(n,r)))return new u(p(i));s=this,o=G[1];for(;;){r&!0&&(o=o.times(s),--r);if(r===0)break;r/=2,s=s.square()}return o},u.prototype.pow=o.prototype.pow,o.prototype.modPow=function(e,t){e=Q(e),t=Q(t);if(t.isZero())throw new Error("Cannot take modPow with modulus 0");var n=G[1],r=this.mod(t);if(r.isZero())return G[0];while(e.isPositive())e.isOdd()&&(n=n.multiply(r).mod(t)),e=e.divide(2),r=r.square().mod(t);return n},u.prototype.modPow=o.prototype.modPow,o.prototype.compareAbs=function(e){var t=Q(e),n=this.value,r=t.value;return t.isSmall?1:O(n,r)},u.prototype.compareAbs=function(e){var t=Q(e),n=Math.abs(this.value),r=t.value;return t.isSmall?(r=Math.abs(r),n===r?0:n>r?1:-1):-1},o.prototype.compare=function(e){if(e===Infinity)return-1;if(e===-Infinity)return 1;var t=Q(e),n=this.value,r=t.value;return this.sign!==t.sign?t.sign?1:-1:t.isSmall?this.sign?-1:1:O(n,r)*(this.sign?-1:1)},o.prototype.compareTo=o.prototype.compare,u.prototype.compare=function(e){if(e===Infinity)return-1;if(e===-Infinity)return 1;var t=Q(e),n=this.value,r=t.value;return t.isSmall?n==r?0:n>r?1:-1:n<0!==t.sign?n<0?-1:1:n<0?1:-1},u.prototype.compareTo=u.prototype.compare,o.prototype.equals=function(e){return this.compare(e)===0},u.prototype.eq=u.prototype.equals=o.prototype.eq=o.prototype.equals,o.prototype.notEquals=function(e){return this.compare(e)!==0},u.prototype.neq=u.prototype.notEquals=o.prototype.neq=o.prototype.notEquals,o.prototype.greater=function(e){return this.compare(e)>0},u.prototype.gt=u.prototype.greater=o.prototype.gt=o.prototype.greater,o.prototype.lesser=function(e){return this.compare(e)<0},u.prototype.lt=u.prototype.lesser=o.prototype.lt=o.prototype.lesser,o.prototype.greaterOrEquals=function(e){return this.compare(e)>=0},u.prototype.geq=u.prototype.greaterOrEquals=o.prototype.geq=o.prototype.greaterOrEquals,o.prototype.lesserOrEquals=function(e){return this.compare(e)<=0},u.prototype.leq=u.prototype.lesserOrEquals=o.prototype.leq=o.prototype.lesserOrEquals,o.prototype.isEven=function(){return(this.value[0]&1)===0},u.prototype.isEven=function(){return(this.value&1)===0},o.prototype.isOdd=function(){return(this.value[0]&1)===1},u.prototype.isOdd=function(){return(this.value&1)===1},o.prototype.isPositive=function(){return!this.sign},u.prototype.isPositive=function(){return this.value>0},o.prototype.isNegative=function(){return this.sign},u.prototype.isNegative=function(){return this.value<0},o.prototype.isUnit=function(){return!1},u.prototype.isUnit=function(){return Math.abs(this.value)===1},o.prototype.isZero=function(){return!1},u.prototype.isZero=function(){return this.value===0},o.prototype.isDivisibleBy=function(e){var t=Q(e),n=t.value;return n===0?!1:n===1?!0:n===2?this.isEven():this.mod(t).equals(G[0])},u.prototype.isDivisibleBy=o.prototype.isDivisibleBy,o.prototype.isPrime=function(){var t=M(this);if(t!==e)return t;var n=this.abs(),r=n.prev(),i=[2,3,5,7,11,13,17,19],s=r,o,u,a,f;while(s.isEven())s=s.divide(2);for(a=0;a<i.length;a++){f=bigInt(i[a]).modPow(s,n);if(f.equals(G[1])||f.equals(r))continue;for(u=!0,o=s;u&&o.lesser(r);o=o.multiply(2))f=f.square().mod(n),f.equals(r)&&(u=!1);if(u)return!1}return!0},u.prototype.isPrime=o.prototype.isPrime,o.prototype.isProbablePrime=function(t){var n=M(this);if(n!==e)return n;var r=this.abs(),i=t===e?5:t;for(var s=0;s<i;s++){var o=bigInt.randBetween(2,r.minus(2));if(!o.modPow(r.prev(),r).isUnit())return!1}return!0},u.prototype.isProbablePrime=o.prototype.isProbablePrime,o.prototype.next=function(){var e=this.value;return this.sign?b(e,1,this.sign):new o(m(e,1),this.sign)},u.prototype.next=function(){var e=this.value;return e+1<r?new u(e+1):new o(i,!1)},o.prototype.prev=function(){var e=this.value;return this.sign?new o(m(e,1),!0):b(e,1,this.sign)},u.prototype.prev=function(){var e=this.value;return e-1>-r?new u(e-1):new o(i,!0)};var _=[1];while(_[_.length-1]<=t)_.push(2*_[_.length-1]);var D=_.length,P=_[D-1];o.prototype.shiftLeft=function(e){if(!H(e))return e.isNegative()?this.shiftRight(e.abs()):this.times(G[2].pow(e));e=+e;if(e<0)return this.shiftRight(-e);var t=this;while(e>=D)t=t.multiply(P),e-=D-1;return t.multiply(_[e])},u.prototype.shiftLeft=o.prototype.shiftLeft,o.prototype.shiftRight=function(e){var t;if(!H(e))return e.isNegative()?this.shiftLeft(e.abs()):(t=this.divmod(G[2].pow(e)),t.remainder.isNegative()?t.quotient.prev():t.quotient);e=+e;if(e<0)return this.shiftLeft(-e);var n=this;while(e>=D){if(n.isZero())return n;t=A(n,P),n=t[1].isNegative()?t[0].prev():t[0],e-=D-1}return t=A(n,_[e]),t[1].isNegative()?t[0].prev():t[0]},u.prototype.shiftRight=o.prototype.shiftRight,o.prototype.not=function(){return this.negate().prev()},u.prototype.not=o.prototype.not,o.prototype.and=function(e){return B(this,e,function(e,t){return e&t})},u.prototype.and=o.prototype.and,o.prototype.or=function(e){return B(this,e,function(e,t){return e|t})},u.prototype.or=o.prototype.or,o.prototype.xor=function(e){return B(this,e,function(e,t){return e^t})},u.prototype.xor=o.prototype.xor;var j=1<<30,F=(t&-t)*(t&-t)|j,X=function(e,t){var n=G[0],r=G[1],i=e.length;if(2<=t&&t<=36&&i<=s/Math.log(t))return new u(parseInt(e,t));t=Q(t);var o=[],a,f=e[0]==="-";for(a=f?1:0;a<e.length;a++){var l=e[a].toLowerCase(),c=l.charCodeAt(0);if(48<=c&&c<=57)o.push(Q(l));else if(97<=c&&c<=122)o.push(Q(l.charCodeAt(0)-87));else{if(l!=="<")throw new Error(l+" is not a valid character");var h=a;do a++;while(e[a]!==">");o.push(Q(e.slice(h+1,a)))}}o.reverse();for(a=0;a<o.length;a++)n=n.add(o[a].times(r)),r=r.times(t);return f?n.negate():n};o.prototype.toString=function(t){t===e&&(t=10);if(t!==10)return $(this,t);var n=this.value,r=n.length,i=String(n[--r]),s="0000000",o;while(--r>=0)o=String(n[r]),i+=s.slice(o.length)+o;var u=this.sign?"-":"";return u+i},u.prototype.toString=function(t){return t===e&&(t=10),t!=10?$(this,t):String(this.value)},o.prototype.valueOf=function(){return+this.toString()},o.prototype.toJSNumber=o.prototype.valueOf,u.prototype.valueOf=function(){return this.value},u.prototype.toJSNumber=u.prototype.valueOf;var G=function(e,t){return typeof e=="undefined"?G[0]:typeof t!="undefined"?+t===10?Q(e):X(e,t):Q(e)};for(var Y=0;Y<1e3;Y++)G[Y]=new u(Y),Y>0&&(G[-Y]=new u(-Y));return G.one=G[1],G.zero=G[0],G.minusOne=G[-1],G.max=q,G.min=R,G.gcd=U,G.lcm=z,G.isInstance=function(e){return e instanceof o||e instanceof u},G.randBetween=W,G}();typeof module!="undefined"&&module.hasOwnProperty("exports")&&(module.exports=bigInt);

function compute(){
  var base = bigInt(document.getElementById('base').value);
  var exponent = bigInt(document.getElementById('exponent').value);
  var modulo = bigInt(document.getElementById('modulo').value);

  var res = base.modPow(exponent, modulo);
  document.getElementById('result').value = res;
}
Base:<input id="base" value="3">

Exponent:<input id="exponent" value="23984081230374123749712934791249172394719237497219347129374498">

Modulo:<input id="modulo" value="112457129983317064494133258034491756790943511028023366901014968560410379195027">

<input type="button" value="Compute" onclick="compute()">

Result: <input id="result">

The result seems to be exact.

PS: The modPow implementation is actually quite readable.


Structure to ByteArray Extension I have a need to turn various structures into byte arrays to be sent over serial port to another machine. I created generic extensions to turn any structure into a byte array and from a byte array back into a given structure.

I've not used the Marshal class often, and I'm uncomfortable manipulating memory on this level.

Do I need to explicitly free any memory?
Is this an efficient implementation? I'll be calling this a lot, it's important that it's efficient in both terms of memory and speed.

public static class StructureExtensions
{
    public static byte[] ToByteArray<T>(this T structure) where T : struct
    {
        var bufferSize = Marshal.SizeOf(structure);
        var byteArray = new byte[bufferSize];

        IntPtr handle = Marshal.AllocHGlobal(bufferSize);
        Marshal.StructureToPtr(structure, handle, true);
        Marshal.Copy(handle, byteArray, 0, bufferSize);

        return byteArray;
    }

    public static T ToStructure<T>(this byte[] byteArray) where T : struct
    {
        var packet = new T();
        var bufferSize = Marshal.SizeOf(packet);
        IntPtr handle = Marshal.AllocHGlobal(bufferSize);
        Marshal.Copy(byteArray, 0, handle, bufferSize);

        return Marshal.PtrToStructure<T>(handle);
    }
}

  Example Call:

var point = new Point(10,5);
byte[] serialized = point.ToByteArray();

Point deserialized = serialized.ToStructure<Point>();

I haven't used it, however all of the documentation seems to suggest that you need explicitly free the memory allocated through:

IntPtr handle = Marshal.AllocHGlobal(bufferSize);

With a paired call to:

Marshal.FreeHGlobal(handle);

You don't seem to need the handle after you've performed the marshalling, so it could be as simple as updating the function to:

public static byte[] ToByteArray<T>(this T structure) where T : struct
{
    var bufferSize = Marshal.SizeOf(structure);
    var byteArray = new byte[bufferSize];

    IntPtr handle = Marshal.AllocHGlobal(bufferSize);
    try
    {
        Marshal.StructureToPtr(structure, handle, true);
        Marshal.Copy(handle, byteArray, 0, bufferSize);

    }
    finally
    {
        Marshal.FreeHGlobal(handle);
    }
    return byteArray;
}

Depending on the overhead of The AllocHGlobal / FreeHGlobal and if you're going to be marshalling the same structures over and over again you might be better off going with something like:

public sealed class Marshaller<T> : IDisposable where T : struct
{
    readonly IntPtr _handle;
    readonly int _bufferSize;

    public Marshaller()
    {
        _bufferSize = Marshal.SizeOf(typeof(T));
        _handle = Marshal.AllocHGlobal(_bufferSize);
    }

    public byte[] ToByteArray(T structure)
    {
        var byteArray = new byte[_bufferSize];

        Marshal.StructureToPtr(structure, _handle, true);
        Marshal.Copy(_handle, byteArray, 0, _bufferSize);

        return byteArray;
    }

    public T ToStructure(byte[] byteArray)
    {
        var packet = new T();

        Marshal.Copy(byteArray, 0, _handle, _bufferSize);

        return Marshal.PtrToStructure<T>(_handle);
    }

    #region IDisposable Support
    private bool disposedValue = false;

    ~Marshaller() {
       Dispose();
    }

    public void Dispose()
    {
        if (!disposedValue)
        {
            Marshal.FreeHGlobal(_handle);

            disposedValue = true;
        }

        GC.SuppressFinalize(this);
    }
    #endregion

}


Why is my web scraping script so slow? My script, which is a web scraping script, is very slow, I even needed to put set_time_limit(0);

This is the whole http://phpfiddle.org/main/code/9qt-78n

I think the problem is here:

foreach ($array_with_links as $url_job) {

$info=Array(getTitle($url_job, getID($url_job)), getTitle_Short($url_job, getID($url_job)), getCity($url_job),
 getDepartment($url_job), getSalary($url_job), getJobNumber($url_job), getPositionStartDate($url_job), getFullTimeEquivalent($url_job),
  getPermTermCasual($url_job), getLocation($url_job), getQualifications($url_job), getDuties($url_job),
  getClosingDate($url_job), getContact($url_job), getEmail($url_job), getCreated_On($url_job), getID($url_job) );

array_push($data, $info);}

Example of some of those functions:

function getCity($url)
    {
    $url = curl_get_contents($url);
    $html_object = str_get_html($url);
    return $ret = $html_object->find('td', 86)->plaintext;
    }

function getDepartment($url)
    {
    $url = curl_get_contents($url);
    $html_object = str_get_html($url);
    return $ret = $html_object->find('td', 90)->plaintext;
    }

And this is my cURL funtion:

function curl_get_contents($url)
{
  $curl_moteur = curl_init();
  curl_setopt($curl_moteur, CURLOPT_URL, $url);
  curl_setopt($curl_moteur, CURLOPT_RETURNTRANSFER, 1);

  curl_setopt($curl_moteur,CURLOPT_USERAGENT,'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.13) Gecko/20080311 Firefox/2.0.0.13');

  curl_setopt($curl_moteur, CURLOPT_FOLLOWLOCATION, 1);
  $web = curl_exec($curl_moteur);
  curl_close($curl_moteur);
  return $web;
}

Those getX come from an URL one by one.  Maybe is there any method to make multiple simultaneous insertions in that array?

I really don't know what to do and what's my mistake.

It seems that every one of your getCity(), getDepartment() etc functions loads the same web page over and over.

You should load each URL once with your curl_get_contents(), then pass its result into each get*() function to parse it.


String case reverse function in C++ This reverses the case of a character in a string (a string being an array of characters). How does it look? Anything I can improve on? Let's get critical here.

#include <iostream>
#include <string>
//
// return a new string
//
std::string camel_case_rev(const std::string &word)
{
        std::string ret = "";

        for(unsigned i = 0; i < word.size(); i++) {
                if(word[i] <= 'Z' && word[i] >= 'A') {
                        ret += tolower(word[i]);
                }
                else if(word[i] <= 'z' && word[i] >= 'a') {
                        ret += toupper(word[i]);
                }
                else {
                        ret += word[i];
                }
        }
        return ret;
}

int main(int argc, char **argv)
{
        if(argc < 2) {
                std::cerr << "USAGE: " << argv[0] << " " << "<string>\n";
                return 1;
        }

        std::string a = camel_case_rev(argv[1]);

        std::cout << a << "\n";

        return 0;
}

This seems like a perfect opportunity to leverage the standard library. I would make a function to toggle a single character and then use std::transform on top of that. Not only is it more consise, it will be more performant (you should use reserve to preallocate the resulting string -- otherwise the string may allocate over and over again).

#include <cctype>
#include <string>
#include <algorithm>

int toggle_case(int chr) {
    return std::islower(chr) ? std::toupper(chr) : std::tolower(chr);
}

std::string toggle_case(std::string str) {
    std::transform(std::begin(str), std::end(str), std::begin(str), toggle_case);
    return str;
}

Note that standard islower and isupper were used instead of comparison against character literals. Not only is it more compact and clearer, C++ does not guarantee that 'A', 'B', ..., 'Z' are contiguous.

Also, since you're (presumably) including the C++ version of ctype.h, cctype, you should be using std::toupper and std::tolower. C++ versions of C headers are allowed to place symbols in the global namespace, but they're only required to place them in std. There's no existing compiler I know of that doesn't also place them in the global namespace, but it can never hurt to strive for portabiltiy, especially when it's easy.

return std::islower(chr) ? std::toupper(chr) : std::tolower(chr);

This might look a little weird, but std::tolower and std::toupper are guaranteed to have no effect if no opposite-cased character exists for the input. In other words, std::tolower(c) is guaranteed to have no effect unless std::toupper(c) is true. To me, this makes sense and this short-cutted logic seems fairly natural. I could see this being rather weird to read though, so it might be better as a more explicit conditional chain.

I should also mention that the code makes an implicit copy of the string via the parameter. If you wanted to make the copy more explicit, you could, but that would actually be potentially hampering performance since it would disable movement into the parameter. In short, with the implicit copy, there's a potential that a copy will not actually happen. With an explicit copy, a copy will always happen.


Can anyone improve my multidimensional array more distinctive? What I've been working on is some kind of function that can take basically any well constructed multi-dimensional array and remove any and all duplicates within matching key sets.

The particular function I've designed provides convenience and functionality but it is far from pretty and seemingly slow compared to a possibly more optimized function.

Sample Data

$echo['level-one'][] = '1';
$echo['level-one'][] = '1';
$echo['level-one'][] = '1';
$echo['level-one'][] = '1';
$echo['level-one']['level-two'][] = '1';
$echo['level-one']['level-two'][] = '1';
$echo['level-one']['level-two'][] = '1';
$echo['level-one']['level-two'][] = '1';
$echo['level-one']['level-two']['level-three'][] = '1';
$echo['level-one']['level-two']['level-three'][] = '2';
$echo['level-one']['level-two']['level-three'][] = '3';
$echo['level-one']['level-two']['level-three'][] = '4';

Function Declaration and Execution

function array_unique_multidimensional($name=null,$tv=null,$tk=null,$ta=null){

  if(!isset($ta)){$ta = ''.$name.'';}else{$ta .= '['.$tk.']';}

  if(!isset($tv)){$tv = '$v';}else{$tv .= 'v';}

  if(!isset($tk)){$tk = '$k';}else{$tk .= 'k';}

  return('
    foreach('.$ta.' as '.$tk.' => '.$tv.'){

      if(is_array('.$tv.')){
        eval(array_unique_multidimensional(\'\',\''.$tv.'\',\''.$tk.'\',\''.$ta.'\'));
      }else{
        '.$ta.' = array_unique('.$ta.');
      }

     }
  '); //end return

}

eval(array_unique_multidimensional('$echo'));
//Yes this function returns a foreach loop so it must be eval'd
//then it recursively calls itself until the entire array is traversed
//This will ensure every segment of the array is unique individually

Alright so I spaced it out a lot more then it probably needs to be and it is probably best viewed in some kind of program editor. Anyway after running the script as it is shown above you should get level's one and two reduced to a single entry and level three should have all four of it's entries (since their values are unique and one and two's are not).

I've seen a lot of answers using array_map to serialize and unique but I've found that they do not work for multidimensional arrays of varying calibers. Ultimately this function will accept any size and depth of array, compare every single key set to itself, remove any duplicates, and preserve every array name/key contained within.

Does anyone know of a better way to go about this? If not enjoy the working code above, if you do please share :)

Shortest solution I can come up with:

function array_unique_recursive ($array) {

    // First loop the array and call self on any sub arrays
    // Note that $value is referenced
    foreach ($array as &$value) {
        if (is_array($value)) {
            $value = array_unique_recursive($value);
        }
    }

    // Now unique this level
    return array_unique($array, SORT_REGULAR);

}

print_r(array_unique_recursive($echo));

See it working


Crop black border of image using NumPy Objective: Crop the image so only the number stays in the image

Problem: Slow Performance

I have code that crops an image.
The image pixels are 0 or 255. There are no values between.

The background is 0 (black) and the letter/number is between 0 (not-inclusive) - 255 (white).

This code is being used to crop a Mnist Digit of the Mnist Dataset.

Image Source

The code does it, however, it does with 2 fors and it takes a long time! How can I optimize it?

def crop_image(data_to_crop):
    cropped_data = []

    for z in xrange(data_to_crop.shape[0]):

        img = data_to_crop[z]
        img = img.reshape(data_to_crop.shape[0], data_to_crop.shape[1])

        rx = -1
        upy = -1

        lx = -1
        by = -1

        for x in xrange(data_to_crop.shape[0]):
            for y in xrange(data_to_crop.shape[1]):

                px = img[x, y]

                if px > 0:

                    if rx == -1 or x > rx:
                        rx = x

                    if lx == -1 or x < lx:
                        lx = x

                    if upy == -1 or y > upy:
                        upy = y

                    if by == -1 or y < by:
                        by = y

        img = img[lx:rx, by:upy]

        cropped_data.append(img)

    return cropped_data

Vectorization with NumPy
When read with cv2.imread or skimage.io.imread or scipy.misc.imread, you would already have the image data as a NumPy array. Now, NumPy supports various vectorization capabilities, which we can use to speed up things quite a bit.
I. Crop to remove all black rows and columns across entire image
To solve our case, one method would be to look for rows and columns that have at least one pixel along rows and columns that is greater than some lower limit or threshold as a pixel value. So, if you are sure that the black areas are absolutely zeros, you can set that threshold as 0. Thus, if img represents the image data, you would have correspondingly two boolean arrays : (img>tol).any(1) and (img>tol).any(0).
Next up, you can use these boolean arrays to index into the image data for extraction of valid bounding box using broadcasted indexing with np._ix -
np.ix_((img>tol).any(1),(img>tol).any(0))

Finally, we index into image data with it for the final extracted data, which is the required bounding box data.
To sum up, the final implementation would be -
def crop_image(img,tol=0):
    # img is 2D image data
    # tol  is tolerance
    mask = img>tol
    return img[np.ix_(mask.any(1),mask.any(0))]

Sample results
1] With tolerance = 0 :

2] With tolerance = 80 (tighter) :

II. Crop while keeping the inner all black rows or columns
To crop the image while keeping the inner all black rows or columns, the implementation would be close to the previous method. The basic idea here would be getting the start, stop indices along rows and columns that decide the bounding box. We will start off with the same mask of ANY match along rows and columns as used in the previous one. Then, argmax would be useful to get start and stop indices of those matches along rows and cols. We will use these indices to slice the 2D input array, which is the desired cropped image output. The implementation would look something like this -
def crop_image_only_outside(img,tol=0):
    # img is 2D image data
    # tol  is tolerance
    mask = img>tol
    m,n = img.shape
    mask0,mask1 = mask.any(0),mask.any(1)
    col_start,col_end = mask0.argmax(),n-mask0[::-1].argmax()
    row_start,row_end = mask1.argmax(),m-mask1[::-1].argmax()
    return img[row_start:row_end,col_start:col_end]

Sample run -
Input :

Output :

Benchmarking
Since we are talking about performance in this Q&A, let's test out how this method works in comparison to others.
@Gareth Rees's solution is another vectorized one that finds all indices and then slices the input array. Finding all indices could become costly. We will try to time and see how much would that affect the performance. we will use the sample data used in the earlier section -
# @Gareth Rees's solution
def crop_with_argwhere(image):
    # Mask of non-black pixels (assuming image has a single channel).
    mask = image > 0

    # Coordinates of non-black pixels.
    coords = np.argwhere(mask)

    # Bounding box of non-black pixels.
    x0, y0 = coords.min(axis=0)
    x1, y1 = coords.max(axis=0) + 1   # slices are exclusive at the top

    # Get the contents of the bounding box.
    cropped = image[x0:x1, y0:y1]
    return cropped

Timings -
# Import the "4" digit image from previous section
In [50]: from skimage import io
    ...: im = io.imread('https://i.stack.imgur.com/tmdiH.png')

# @Gareth Rees's solution
In [51]: %timeit crop_with_argwhere(im)
1000 loops, best of 3: 1.4 ms per loop

In [52]: %timeit crop_image_only_outside(im,tol=0)
10000 loops, best of 3: 81.8 µs per loop

The memory efficiency with crop_image_only_outside is noticeable on performance.

Extend to generic 2D or 3D image data cases
Assuming we are looking to check for ALL matches across all channels along the last dimension/axis, the extension would be simply performing numpy.all reduction along the last axis. Hence, we would have generic solutions to handle both 2D and 3D image data cases like so -
def crop_image(img,tol=0):
    # img is 2D or 3D image data
    # tol  is tolerance
    mask = img>tol
    if img.ndim==3:
        mask = mask.all(2)
    mask0,mask1 = mask.any(0),mask.any(1)
    return img[np.ix_(mask0,mask1)]

def crop_image_only_outside(img,tol=0):
    # img is 2D or 3D image data
    # tol  is tolerance
    mask = img>tol
    if img.ndim==3:
        mask = mask.all(2)
    m,n = mask.shape
    mask0,mask1 = mask.any(0),mask.any(1)
    col_start,col_end = mask0.argmax(),n-mask0[::-1].argmax()
    row_start,row_end = mask1.argmax(),m-mask1[::-1].argmax()
    return img[row_start:row_end,col_start:col_end]


Draw element border according to scroll amount I needed to draw border element according to scroll amount.
I made it work but I am not sure about optimization and performance.
There may have a better way? I am open to suggestion

you can see it in action here: enter link description here

here my js:

Thanks

var el = document.querySelectorAll(".animateBorder");

document.addEventListener("scroll", (event) => {
    for (var i = 0; i < el.length; i++) {
        var mHeight = window.innerHeight / 2 || document.documentElement.clientHeight / 2;
        var currentPos = document.documentElement.scrollTop + mHeight;

        var elTop = el[i].offsetTop;
        var elHeight = el[i].offsetHeight;
        var count = el[i].dataset.count;

        var partLenght = elHeight / count;

        var elEnd = elTop + elHeight;

        var spans = el[i].getElementsByTagName("span");

        for (var s = 0; s < spans.length; s++) {
            var offset = s * partLenght;
            var _0 = elTop + offset;
            var _100 = _0 + partLenght;
            var direction = spans[s].dataset.direction;
            // console.log("s", s);
            // console.log("partLenght", partLenght);
            // console.log("elTop", elTop);
            // console.log("offset", offset);
            // console.log("currentPos", currentPos);
            // console.log("_0", _0);
            // console.log("_100", _100);
            if (_0 <= currentPos && currentPos <= _100) {
                console.log("spans[s]", s);
                console.log("direction", direction);
                if (direction == "horizontal") {
                    spans[s].style.width = Math.round(((currentPos - _0) * 100) / partLenght) + "%";
                }
                if (direction == "vertical") {
                    spans[s].style.height = Math.round(((currentPos - _0) * 100) / partLenght) + "%";
                }
            }
            if (currentPos >= _100) {
                if (direction == "horizontal") {
                    spans[s].style.width = "100%";
                }
                if (direction == "vertical") {
                    spans[s].style.height = "100%";
                }
            }

            if (currentPos <= _0) {
                if (direction == "horizontal") {
                    spans[s].style.width = "0%";
                }
                if (direction == "vertical") {
                    spans[s].style.height = "0%";
                }
            }
        }
    }
});

When you have a collection that you want to iterate over, and you don't particularly care about the index of each element, iterating by going through the collection's .length and manually incrementing an index counter anyway is a bit tedious and ugly. Collections almost always have iterators or a forEach method - it's nice to use those instead, when possible. In other words, rather than

var el = document.querySelectorAll(".animateBorder");
for (var i = 0; i < el.length; i++) {
  // lots of lines referencing el[i]

you may use

const borderContainers = document.querySelectorAll(".animateBorder");
for (const borderContainer of borderContainers) {
  // lots of lines referencing borderContainer

Don't call the collection el - I, and most other developers, would ordinarily expect a variable named el to contain an element. Since what you have is a collection of elements, it would be good to be precise and make that clearer with the variable name, else a script reader or writer is more likely to get confused by it and make a mistake as a result.

You define the event argument, but never use it:

document.addEventListener("scroll", (event) => {

If you're not going to use an argument, it would be better to leave it out entirely, that way there's one less variable readers of the script have to be mindful of:

document.addEventListener("scroll", () => {

scroll events can sometimes fire very frequently (like 50 in a second or two):

const div = document.querySelector('div');
window.addEventListener('scroll', () => {
  div.textContent = Number(div.textContent) + 1;
});
body {
  height: 2000px;
}

0

Given these sorts of events, and given that you might be iterating over a non-trivial number of animateBorder elements, and each of those animateBorder's descendant s, you might be right to consider the performance implications of your code.

One way to improve would be to retrieve values only when necessary. For example, rather than doing window.innerHeight / 2 || document.documentElement.clientHeight / 2; for every scroll, and also for every animateBorder item that exists during that scroll event, you could retrieve the value just once, on pageload. You can also retrieve the scrollTop when a scroll event occurs, rather than for every animateBorder element in a scroll event. Another option would be to add a small debouncer - only iterate through the elements and change their styles once, say, 50ms has passed since the last style change has taken place.

You define

partLenght

Spelling matters in programming. Misspelled words are a very easy source of bugs. Use partLength instead.

Variables named _0 and _100 are pretty weird - it doesn't look like a normal variable name, but almost like a numeric literal with the new _ optional separators, eg:

console.log(
  1_000_000
);

Better to call the variables something easily understandable with English words instead, maybe spanTopOffset and spanBotOffset.

You have

if (direction == "horizontal") {

and a few other uses of == as well. Always use strict equality with === and !== instead - otherwise, if you use loose equality, you'll be requiring everyone that reads the code (including you) to be sure of all of the weird type coercion rules invoked by loose equality.

Since it's 2020, best to write at least in ES2015 syntax (which you're already partially doing with =>). If you're going to use ES2015 syntax, use it everywhere - never use var, since it has too many problems. Define variables with const instead (or let if you have no choice but to reassign the variable).

To clarify the logic in the comparisons against currentPos, consider using else-if and else instead, to make it clear that exactly one of the conditions may be fulfilled. I'd also add comments indicating what exactly each condition means.

Or, an alternative would be to simply calculate the percentage required regardless, then constrain its values to between 0 and 100 - then set the width or height to that constrained percentage:

// 0 when current position before or at top of span section
// between 0 and 100 when current position is between top and bottom of span section
// 100 when current position is after bottom of span section
const percentScrolledThisElement = Math.min(
  100,
  Math.max(
    0,
    Math.round(((currentPos - spanTopOffset) * 100) / partLength)
  )
);
if (direction === "horizontal") {
  span.style.width = percentScrolledThisElement + "%";
} else {
  span.style.height = percentScrolledThisElement + "%";
}

const borderContainers =  document.querySelectorAll(".animateBorder");
// If screen size changes, you can add a resize event to reassign this:
const mHeight = window.innerHeight / 2 || document.documentElement.clientHeight / 2;
document.addEventListener("scroll", () => {
  const currentPos = document.documentElement.scrollTop + mHeight;
  for (const borderContainer of borderContainers) {
  const { offsetTop, offsetHeight } = borderContainer;
    const { count } = borderContainer.dataset;
  const partLength = offsetHeight / count;
  const elEnd = offsetTop + offsetHeight;
    borderContainer.querySelectorAll('span').forEach((span, i) => {
   const offset = i * partLength;
   const spanTopOffset = offsetTop + offset;
   const spanBotOffset = spanTopOffset + partLength;
   const { direction } = span.dataset;

      // 0 when current position before or at top of span section
      // between 0 and 100 when current position is between top and bottom of span section
      // 100 when current position is after bottom of span section
      const percentScrolledThisElement = Math.min(
        100,
        Math.max(
          0,
          Math.round(((currentPos - spanTopOffset) * 100) / partLength)
        )
      );
      if (direction === "horizontal") {
        span.style.width = percentScrolledThisElement + "%";
      } else {
        span.style.height = percentScrolledThisElement + "%";
      }
  });
 }
});
#lipsum {
  max-width: 680px;
  margin: 0 auto;
}

.animateBorder {
  display: flex;
  outline: 1px solid red;
  height: 300px;
  width: 50%;
  align-items: center;
  justify-content: center;
  position: relative;
}

span {
  display: block;
  background-color: green;
  position: absolute;
}
span.b-top {
  top: 0;
  width: 0;
  height: 2px;
}
span.b-top.d-right {
  left: 0;
}
span.b-top.d-left {
  right: 0;
}
span.b-right {
  right: 0%;
  height: 0;
  width: 2px;
}
span.b-right.d-up {
  bottom: 0;
}
span.b-right.d-down {
  top: 0%;
}
span.b-bottom {
  top: 100%;
  width: 0;
  height: 2px;
}
span.b-bottom.d-right {
  left: 0;
}
span.b-bottom.d-left {
  right: 0;
}
span.b-left {
  left: 0%;
  height: 0;
  width: 2px;
}
span.b-left.d-up {
  bottom: 0;
}
span.b-left.d-down {
  top: 0%;
}

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Nam lacinia mi eget efficitur pretium. Aliquam neque mauris, commodo sed leo quis, rutrum rhoncus dui. Nulla in sollicitudin magna. Aliquam at fermentum metus. Integer ac diam id urna placerat ornare. Integer consequat condimentum sem, quis finibus mi hendrerit non. Morbi quis fringilla purus, consequat imperdiet mauris. Sed dictum pretium tellus, eget euismod sem.

Morbi suscipit magna nibh, vel tempus justo fermentum non. Nullam metus sapien, viverra quis viverra ac, ullamcorper ut neque. Maecenas eget orci quis libero aliquam gravida id in ligula. Mauris molestie consectetur erat, sed pharetra magna. Fusce consequat luctus libero sit amet posuere. Aliquam ultricies purus a nunc molestie elementum. Aliquam erat volutpat.

  top,right,bottom

  left,bottom

Nam at ligula leo. Integer eu ante vitae nisi pellentesque feugiat in vitae neque. Vestibulum ante ipsum primis in faucibus orci luctus et ultrices posuere cubilia curae; Lorem ipsum dolor sit amet, consectetur adipiscing elit. Vestibulum ante odio, placerat eu tincidunt in, luctus a diam. Aenean sed elit nunc. Lorem ipsum dolor sit amet, consectetur adipiscing elit. Curabitur a ullamcorper orci. Vestibulum ante ipsum primis in faucibus orci luctus et ultrices posuere cubilia curae; Proin finibus elit sed urna tempus, in volutpat ipsum volutpat.

Praesent nunc elit, aliquam id sagittis in, ornare a dui. Vestibulum aliquam sem sit amet risus porttitor, ornare fringilla ex tristique. Vivamus quis augue id enim placerat finibus. In ullamcorper, tortor eu finibus euismod, sapien felis eleifend urna, feugiat convallis metus purus sed ante. Pellentesque volutpat egestas mi, in luctus mauris. Suspendisse pellentesque sed enim eu maximus. Donec vestibulum orci nunc, sed egestas mauris iaculis vel. Aenean malesuada bibendum turpis eget lobortis. Sed et ex non lorem varius interdum. Class aptent taciti sociosqu ad litora torquent per conubia nostra, per inceptos himenaeos. Praesent ultrices vehicula pretium. Aliquam mollis dolor ut commodo viverra. Nam nibh mauris, hendrerit et ipsum sed, aliquet auctor lacus. Maecenas tincidunt neque sed enim facilisis sagittis.

  top,right,bottom

Ut ornare, metus eu consectetur pulvinar, mi ipsum egestas nibh, et rhoncus odio nulla quis massa. Pellentesque habitant morbi tristique senectus et netus et malesuada fames ac turpis egestas. Duis tincidunt turpis nunc, vitae pretium velit condimentum non. Nunc eleifend ipsum porttitor ligula mattis faucibus. Etiam ac neque vitae nibh finibus lobortis. Morbi laoreet auctor metus, quis sagittis lectus posuere at. Aenean quis gravida lorem. Mauris aliquam, purus nec congue blandit, erat ligula ornare erat, eu malesuada augue nunc in dolor. Mauris vestibulum condimentum suscipit. Suspendisse id nibh sed ex ornare tempus vel vitae velit. Quisque sem enim, sodales nec leo et, fermentum finibus lacus. Suspendisse sit amet rutrum arcu.

Morbi nec efficitur nibh, a condimentum justo. Nunc enim nunc, gravida sed convallis eget, pulvinar nec ex. Pellentesque vitae feugiat massa. Phasellus fermentum orci eu commodo pulvinar. Ut eget porttitor purus. Nam posuere, tortor eleifend condimentum fermentum, lacus quam luctus diam, id congue tortor tortor vel nulla. Vivamus iaculis luctus mauris et dictum. Curabitur rutrum felis eros, vitae tempus risus pellentesque in.

Ut tempor feugiat efficitur. Donec cursus sem consequat malesuada malesuada. Curabitur eu turpis quis tortor aliquet lobortis. Aliquam venenatis, nibh at porttitor faucibus, ex tortor viverra elit, vitae egestas ante augue ut tellus. Phasellus venenatis finibus blandit. Suspendisse sit amet eros vitae odio iaculis accumsan. Pellentesque eu nunc dolor. Vestibulum sed massa at orci ornare bibendum.

Pellentesque nec justo ornare, iaculis turpis ut, vestibulum orci. Integer vel sem sed justo dapibus tempor. Nam hendrerit at sem eu lobortis. Duis id urna ac odio lobortis aliquet. Praesent id porttitor purus. Quisque et nibh in diam accumsan congue ac in velit. Proin libero mauris, interdum sollicitudin mauris id, dignissim ultrices metus. Etiam lacinia augue vitae suscipit euismod. Aliquam pulvinar dui semper lectus accumsan, auctor sagittis erat convallis. Donec faucibus id velit a consectetur.

Suspendisse sodales pretium molestie. Morbi vulputate posuere porta. Duis dictum ultrices facilisis. Sed lobortis libero ipsum, eget tempus turpis blandit sed. Curabitur egestas dolor metus, quis iaculis dolor mattis vitae. Donec ac accumsan leo. Aenean maximus sapien eros.

Praesent leo risus, sagittis ut sapien eget, convallis aliquam lacus. Donec posuere, mauris nec fringilla dictum, purus velit imperdiet purus, vel aliquam magna dui sit amet urna. Phasellus consequat purus id erat ornare congue quis non neque. Donec odio lectus, tincidunt eu sapien sit amet, fringilla dignissim felis. Nunc blandit vehicula ligula quis suscipit. Aenean nec auctor augue. Integer semper mauris eleifend, luctus est sit amet, hendrerit tellus. Donec at erat nec nulla convallis aliquam. Donec laoreet sit amet arcu at pretium. Sed et consequat augue, eget ultricies massa. Suspendisse luctus nulla at eros elementum fringilla.

Duis volutpat erat nec feugiat lacinia. Praesent eget consectetur arcu. Duis fringilla imperdiet libero, eget tincidunt metus tincidunt vel. Suspendisse dignissim quis velit a tempor. Curabitur finibus nec tellus ac aliquet. Cras porttitor efficitur tristique. Maecenas bibendum lacus dui, in mattis tortor aliquam vel. Vestibulum quis tortor semper, consequat nulla eget, lacinia ante. Phasellus ornare vel massa nec eleifend. In aliquet ultricies augue.

Nam ante augue, condimentum non nulla in, imperdiet dignissim augue. Curabitur sodales ullamcorper turpis, vitae convallis lacus vulputate eget. In mollis ipsum eleifend porttitor egestas. Aenean orci mi, scelerisque euismod justo id, luctus hendrerit tortor. Orci varius natoque penatibus et magnis dis parturient montes, nascetur ridiculus mus. In lobortis vitae dui ac efficitur. Nullam iaculis vulputate mi, sed pharetra turpis ultrices non. Pellentesque vitae sodales orci, non vulputate odio. Aliquam eu molestie ex. Maecenas pellentesque lorem est, quis lacinia odio tincidunt nec. Suspendisse pharetra est et magna tincidunt scelerisque.

Vestibulum eu finibus orci. Sed non egestas felis. Vestibulum et nisi vel metus faucibus mattis. Vivamus at ligula sodales, commodo nisl vitae, pharetra tellus. Phasellus quis magna blandit, suscipit magna ac, interdum sem. Cras facilisis dui eu dolor efficitur, sed commodo velit placerat. Sed tempus tortor sit amet nunc dignissim, sit amet vestibulum mi luctus. Donec eget elit ex. Aenean sodales viverra est. Sed accumsan mauris turpis, eu rhoncus lacus volutpat eu. Curabitur et consectetur felis. Nullam at nulla vestibulum, eleifend quam ac, tristique odio. Phasellus sapien ante, blandit ultricies eros at, semper viverra elit.

Sed dictum accumsan efficitur. Sed convallis lobortis porttitor. Nullam vitae lorem sed felis semper tincidunt. Mauris ut ipsum purus. Cras ultricies pharetra tellus sit amet accumsan. Quisque sed nulla lorem. Aliquam pulvinar a nisi sit amet semper. Lorem ipsum dolor sit amet, consectetur adipiscing elit. Nulla aliquam scelerisque erat et facilisis.

Cras ac varius odio, in sollicitudin sapien. Nunc tincidunt vulputate tortor ac euismod. Phasellus dignissim et orci sit amet malesuada. Sed suscipit lectus libero. Maecenas ac aliquam diam, sit amet feugiat nibh. Duis vehicula sem nec tincidunt viverra. Donec non dolor nulla. Curabitur iaculis justo vitae vehicula egestas. Phasellus efficitur volutpat faucibus. Morbi ligula nisl, porttitor nec lectus non, accumsan ornare augue. Nullam imperdiet nisl eu urna bibendum finibus nec vel augue. Phasellus vestibulum dui at interdum sollicitudin. Etiam ac commodo metus. Vestibulum augue metus, commodo vitae felis venenatis, aliquet feugiat mauris. Duis consequat dui enim, in posuere lorem aliquet vitae.