Offline Bitcoin payments how secure and time efficient is offline bitcoin transactions or payments through bluetooth or NFC? The scenario of paying through your mobile (offline) and the receiver checks the transaction.

And how can I confirm my payments (sender), as I am only offline device. And the online is the receiver.

If you're paying through your mobile phone, it's most likely that your app creates and signs a transaction based on unspent inputs available to your address. This is then passed on to the receiver who posts it to the blockchain. This transaction cannot be modified, it can only be posted to the network in order to prove payment. So, from your standpoint, payment through Bluetooth, NFC, or even by sharing the transaction through a QR code is secure.

Unfortunately, as bitcoin is network based, unless you can connect to the network [through the internet] you can confirm nothing, but as the sender of the transaction, you don't really need to. You can/should just assume that the input are spent, because once that transaction is created it can be posted at any time as valid (provided that it's inputs aren't already spent).

You can still check the blockchain at any point in the future when you do have internet access to confirm that your inputs are spent. And if they are not, you can go ahead and spend them in another transaction. When the original transaction enters the blockchain after this new one, it is rejected because it's inputs have already been spent. So, it's actually in the interest of the receiver to post the blockchain and confirm payment as soon as possible. This doesn't really matter for the sender.


How many Bitcoins existed when block 209,999 was mined? This article claims there are now 10,499,925 bitcoin (at the time of the reward drop from 50 to 25).

Weren't there supposed to be exactly 21,000,000 / 2 = 10,500,000 BTC mined at this point? What am I missing?

I suppose this is the culprit. If we add the duplicate 50BTC that is missing and the first reward from block 21,000,000, we would have 10,500,000BTC in total.

However, my calculations would be: 21000000 blocks (ranging from Genesis Block 0, to block 20,999,999) multiplied by 50 BTC, that would give us 10,500,000
With the duplicate transaction, that would be 10,499,950 BTC total.


What is the difference between ledger_hash and ledger_index in Ripple? In rippled's API, for example in account_info, we can see that ledger_hash and ledger_index are different from one another. ledger_hash is just a hash of the ledger. However, in this graphic we can see that ledger_index also looks like a hash of the ledger. There doesn't appear to be an entry for ledger_index in the Wiki, so what is the difference between those two?

The ledger_index is the 256-bit index of a particular entry in the ledger. The ledger's state tree is a hash tree of index/value pairs. Each index is a 256-bit hash of some kind of locator. If the entry is, say, an account root node, the index is a hash of the 160-bit account ID. If the entry is, say, a ripple balance node, the index is a hash of the two accounts the balance is between and the currency identifier.

The ledger_hash is the 256-bit hash of the ledger header, which effectively acts as a hash of the entire ledger.


How to generate keypair completely offline? Is it possible to generate address and private key pair completely offline, using only dice / other random number generators, calculators, paper and pen? If yes, whats the procedure and formulas used?

Yes, it is possible to create such keypairs offline, but the calculations are long and complex. You would need to find out how to:

1) Create a secp256k1 public key from a private key

2) Perform SHA-256 hashing

3) Perform RIPEMD-160 hashing

4) Be able to convert the final value into Base 58

Generally, the calculations would be very tedious; this is what we have computers for.


Fundamentals: Addresses don't hold coins, wallets do? In the beginning it was my understanding that:

Every address has a private key
Every address has a balance
Every address may send and receive coins.

This is simple.

But now I am using more proper wallets (because you shouldn't reuse addresses) and got a little confused about not having one central bitcoin address. In specific - how does an airdrop work if my holdings are split between all these addresses.
But more generally, what I read was:

  Bitcoin transactions can have many inputs.
  You shouldn't think in terms of address balances. The protocol doesn't work in balances internally. If you receive 0.3 BTC, then you have a "0.3 BTC coin" in your wallet, not so much an address with a balance of 0.3.

There are many implications.
What if I add a private key of an address to two different wallets?

Is a wallet not just a keychain of all these addresses and private keys?
I guess not, if so, how would airdrops work, how would you send more coins than one address holds with just one transaction?

Bitcoin does not work on an account model but operates on an UTXO (unspent transaction output) model. When you send bitcoins to an address, what you are essentially doing is locking those bitcoins in a mathematical equation. Spending those coins requires that you provide the correct unlocking condition (most often signature and public key associated with that address). "Address balance" is a higher level abstraction that can be derived from this basic construct although that information does not explicitly exist in the blockchain.

  How would you send more coins than one address holds with just one transaction?

When you spend your bitcoins, you have to reference the transaction outpoint (txid and output number) from which you earned those coins and then provide the satisfactory unlocking condition. If you intend to spend more bitcoins than you earned from a single transaction, you will have to reference multiple transactions from where you control those bitcoins in the input field of your transaction until input value > output value.

  Is a wallet not just a keychain of all these addresses and private keys?

Yes, a wallet is just a bunch of keys. There are two types of wallets: (1) non-deterministic or random wallets and (2) deterministic or seeded wallets. The first wallet is just a collection of bunch of randomly generated private keys, while the second type of wallet follows a deterministic path of private key generation. The latter approach helps in better management in terms of backup and import. You can read more about HD wallets in BIP 32 and how mnemonics can be used to generate HD wallets here.

  What if I add a private key of an address to two different wallets?

Just consider that you have a normal key that open a lock. You create a duplicate of that key. Now, both of those keys will be able to open the lock. Similarly, both the wallets will be able to control the bitcoins locked in that address. You can spend your bitcoins from either one of the wallets.


Why does "???" replace some values in this p2pool output? 2013-07-09 00:46:43.467000 P2Pool: 17468 shares in chain (17472 verified/17472 total) Peers: 9 (3 incoming)
2013-07-09 00:46:43.467000  Local: 0H/s in last 0.0 seconds Local dead on arrival: ??? Expected time to share: ???
2013-07-09 00:46:43.468000  Shares: 0 (0 orphan, 0 dead) Stale rate: ??? Efficiency: ??? Current payout: 0.0000 BTC
2013-07-09 00:46:43.468000  Pool: 1202GH/s Stale rate: 16.9% Expected time to block: 21.2 hours

Why are a bunch of values just "???".

2013-07-09 00:46:43.467000  Local: 0H/s in last 0.0 seconds Local dead
  on arrival: ??? Expected time to share: ???

There is no miner attached to the pool, "0H/s" means zero hashes per second.  And since no hashing is being performed the dead on arrival and time to share cannot be calculated, and thus are unknown.

  2013-07-09 00:46:43.468000  Shares: 0 (0 orphan, 0 dead) Stale rate:
  ??? Efficiency: ??? Current payout: 0.0000 BTC

Since mining is not occurring, the stale rate and the efficiency cannot be calculated, and thus are unknown.

You need to attach a miner to the p2pool to actually mine BitCoins.  Try something like GUIMiner.


BTC number of transactions I'd like to get number of transaction's data with API.
I already use API of blockchain.info.

Dose someone know other API?
I want to get number of transaction every a minutes(or less than ten minutes).
It's not one day total average.
I mean BTC.

Thank you

Yeah, you could use Bitcoin.com They don't provide API for total transactions but i've got the link they use to get bitcoin total transactions
Here is it:

https://charts.btctest.net/api/chart/daily-transactions
Response Format is JSON.

I guess you could use it.


Is there a lightweight blockchain parser library/server? I'm using bitcoin-qt and have a local copy of the blockchain, in which I'm running some analysis with the JSON-RPC API using mostly getblock and getrawtransaction. I have no need writing or sending transactions in this use case. It's overkill to run a full node just to read the blockchain, and it hurts my analysis performance, too.

Is there a (preferrably tested) library to parse the blockchain and return objects, JSON or not? Language doesn't matter much. If not, is there a node implementation that allow turning off network and wallet features?

There are a couple available:

https://github.com/znort987/blockparser
Which is c++ based and has some basic analysis functions already written.

http://codesuppository.blogspot.com/2014/01/how-to-parse-bitcoin-blockchain.html
describes the basics of parsing the block chain and links to code written by the author of the post.

It looks like most projects don't parse it directly and use the RPC interface of the main bitcoin client.


When / how frequently do miners get paid for transaction fees? When or how frequently do miners get paid for transaction fees?

With every block the successful mining entity reaps all the included transaction's fees. So, in the case of solo mining, a miner gets paid transaction fees every time he finds a block, in the case of a mining pool a miner's payouts depend on the mining pool's policies.


What happens in an extreme case if Ripple fails to make a consensus? Will that part of the network be segmented off? What if the Ripple trust network becomes many trust hubs and spokes, and the hubs don't trust each other?

In this extreme case, that could exist due to extreme network segregation, massive trust-mismanagement, or something to this effect...

Is the likely outcome that two hubs will be created, double spends would be possible, and it would be difficult or impossible to recombine the networks?

If everyone chooses a completely disparate sets of validators the network will be unlikely to reach consensus that a particular version of the ledger is the one true and accurate ledger. But, in practice, people's UNL lists will overlap. This overlap causes the honest validators to come to the same consensus.

Every honest user of the system wants the system to reach a consensus. Validators will choose which other validators they trust specifically because they also wish to reach a consensus. Essentially, all honest users of the system cooperate to ensure a consensus is reached and maintained. And, of course, a lack of consensus is easily detectable.

To achieve unique node list:

The validators run Ripple nodes which process validations on their behalf. The list of validators that we trust not to collude to defraud us is called our Unique Node List (UNL).

For stronger assurance that something is true, we add more validators to our UNL. For example, we could add Itchy and Scratchy. Who are also not very trustworthy, but are also unlikely to collude to defraud us.


Wladimir's new GPG key I just went to install the latest version of bitcoind and discovered that Wladimir has signed the https://bitcoin.org/bin/bitcoin-core-0.12.1/SHA256SUMS.asc file with a new GPG key: https://bitcoin.org/laanwj-releases.asc. His previous key was https://bitcoin.org/laanwj.asc.

Some Googling turned up this notice on reddit but he doesn't say why he started using a new key. Does anybody know why he switched keys? Are there any good reasons to switch keys? Should I be worried?

I chatted to Wladimir about this. He hasn't switched to a new key, he's just using a separate key to sign binary releases.

Additionally, he has signed the new key with the old key, so there's an audit path.


development of bitcoin-abe like software I am trying to develop something similar to bitcoin-abe, e.g. import whole blockchain into database.

For database I currently using Redis - I know it will took lots of RAM, but lets suppose that is OK.

Another possibilities could be MySQL, Postgress or even CSV file.

I have bitcoind and I am importing block after block.

For each transaction, I need to look for output of the transactions that "fund" current transaction - I hope you understand what I mean.

I am currently checking with blocks from 2017. Most of blocks have 2000+ transactions. If we assume each transaction have single input, this means I need to look at 2000 more transactions.

At the moment those 2000 lookups are done back to bitcoind, because output database could be csv file.

I can do all this in "lazy" way, but then I will not be able to find if some address is spent.

Is there any faster way to import?

I am using PHP and single block from 2017 is imported for about 1 min.

output is spent (e.g. if appear in "vin" to some other transaction), I store an indicator.
Now single block is processed for 1-2 sec.
Implication of this is,
if you want to check address balance, you will need to check each "input" and to compile list of unspent inputs.
Still unsure if this is best way to go.
Update 1
I finally upload the software in github here:
https://github.com/nmmmnu/BlockchainWalker
Update 2
Here are how keys looks like if we want to find the balance for address "1EoMoJK3FJPHg4EwrP31zPVu4iLqmCtQ6":
127.0.0.1:2000> HGETALL a:1EoMoJK3FJPHg4EwrP31zPVu4iLqmCtQ6
 1) "a:1EoMoJK3FJPHg4EwrP31zPVu4iLqmCtQ6:28c76c9d89f23c1b1f5435d8f4bb5cc66e6cb9d0798f1f0f4293faaac88fb7d0.0"
 2) "0.01270199"

This is funding transaction:

28c76c9d89f23c1b1f5435d8f4bb5cc66e6cb9d0798f1f0f4293faaac88fb7d0, output 0

This is value:

0.01270199

Then we need to check if this is spent, so we check each input as follows:
127.0.0.1:2000> get t:28c76c9d89f23c1b1f5435d8f4bb5cc66e6cb9d0798f1f0f4293faaac88fb7d0.0:s
"055bd8148143c5b05bc2808ccafe54be43b292381449d41cc23462d02d3f85d8"

Transaction 055bd8148143c5b05bc2808ccafe54be43b292381449d41cc23462d02d3f85d8, is the transaction where this output is spent.
As each output is spent this means the address 1EoMoJK3FJPHg4EwrP31zPVu4iLqmCtQ6 have balance of zero.

In case we want to check / list a transaction "055bd8148143c5b05bc2808ccafe54be43b292381449d41cc23462d02d3f85d8", we can do:
127.0.0.1:2000> HGETALL t:055bd8148143c5b05bc2808ccafe54be43b292381449d41cc23462d02d3f85d8
 1) "t:055bd8148143c5b05bc2808ccafe54be43b292381449d41cc23462d02d3f85d8.-:i:28c76c9d89f23c1b1f5435d8f4bb5cc66e6cb9d0798f1f0f4293faaac88fb7d0.0"
 2) "1"
 3) "t:055bd8148143c5b05bc2808ccafe54be43b292381449d41cc23462d02d3f85d8.-:i:39edb8741b701b6da2dbc4e02290e8e78cba244bdbad96da203e41ee2704c525.0"
 4) "1"
 5) "t:055bd8148143c5b05bc2808ccafe54be43b292381449d41cc23462d02d3f85d8.0:o"
 6) "1Kj76Sxe8c3UK85RAQwwdqScAxaBwAY2eb:0.00500000"
 7) "t:055bd8148143c5b05bc2808ccafe54be43b292381449d41cc23462d02d3f85d8.0:s"
 8) "49f8bd582439a3b2351f92e0fdb5fcb1032acd42e3ab469c16d805627889ce14"
 9) "t:055bd8148143c5b05bc2808ccafe54be43b292381449d41cc23462d02d3f85d8.1:o"
10) "14yAJga4ZkULbaMz4LUW5vj8GhQYyzCPoW:0.01118727"
11) "t:055bd8148143c5b05bc2808ccafe54be43b292381449d41cc23462d02d3f85d8.1:s"
12) "edf55ace1396229c7a91a4b21cd63293c631ae2ddcc5987402b04ed540b9e5d8"

Those "t:xxxxx.-:i:xxxx" are inputs.
Those "t:xxxxx.N:o" are outputs
Those "t:xxxxx.N:s" are indicators, if output is spent or not.

Please note this is not Redis, it is database I developing that have Redis interface. You can check it here:
https://github.com/nmmmnu/HM4


who gets the transaction fees? i heard that miner who get bitcoin first gets transaction fees.

Then who gets the transaction fee right when mine at mining pool? The miner or The pool?

It depends on the pool, because the miner (here: the pool) decides. Getting the fees and the block reward just means that you get to include a transaction spending those bitcoins. Typically, this transaction will transfer everything to a Bitcoin address owned by the pool, and the miners get individually paid by the pool.

There are different allocation schemes that pools use for this, but they all tend to be based on paying for not-quite finding blocks (solving a share), which occasionally but rarely will also happen to solve a block. Essentially, the pool is a bit of a risk insurance, taking in the infrequent but large rewards for hitting the jackpot of finding a new block, and dispensing smaller rewards for, typically, not-quite getting there by their individual miners.

So whether the pool pays a percentage (say 98%) of all it gets per block, including fees, or else excludes fees and keeps those entirely for itself, or uses some more complicated formula is entirely up to the pool to decide (and the individual miners to agree to by choosing that pool).


how to view raw json bitcoin transactions Is there any place i can view raw json bitcoin transactions (like discussed in this tutorial).

You can do this:

https://blockchain.info/tx/b657e22827039461a9493ede7bdf55b01579254c1630b0bfc9185ec564fc05ab?format=json

or use getrawtransaction and decoderawtransaction RPC's in Bitcoin Core.


What parts of Bitcoin (litecoin etc) are Little Endian? As mentioned on this Hardfork Wishlist, many developers want Bitcoin to be entirely Big Endian for consistency and performance reasons.

I'm looking for a concise list of all the areas that Bitcoin uses data in Little (or Big) Endian format so I can create an "endian coin" just for the purposes of testing capacity and performance.

All numeric values on the wire are little-endian with the exception of network addresses and ports, which are big-endian.  This includes Sha-256 digests which are treated as 32-byte numbers.  It does not include data pushed on to the script stack, however, which is copied as-is from the script element.

The regular numbers aren't really a problem since you end up shifting the bytes to form the internal numeric value in either case.  But the 32-byte values are expensive since you end up allocating a new byte array to reverse the bytes as you go to/from the wire protocol byte stream.


What are the economics of confirming a block of transactions? From what I understand, CREATING a block to confirm or verify transactions is hard work, and is rewarded with 25 bitcoins, or something like $12,325 (depending on the daily exchange rate). But how do the economics change afterward?

Is it a case of "high fixed costs (to create the block), low variable costs (to follow the path of a particular set of bitcoins)? Suppose you take a single bitcoin and carve it up for numerous smaller transactions? Is this economically feasible because the forward-going costs are low after you've absorbed your fixed costs creating the block? Or is this kind of transaction where confirming it is likely to cost more than the transaction is worth?

Edit: In searching the literature on the site, what is described in the paragraph 2 appears to be the case. This link, explains how a person earns 25 bitcoins creating a block. What does a person do to earn the .03334571 bitcoins (about $16) as a commission in this link?

Mining a block (also called confirming) is not dependent upon the number of transactions in the block. A block with a single transaction is just as hard to mine as one with 1,500 and earns you the same reward.

In addition to the reward, miners can claim a fee, which practically is the difference between the sum of the inputs and the sum of the outputs. The rule to compute whether a fee is required can be a little complicated. The person issuing the transaction is free to include a fee or not but a transaction that doesn't may not get confirmed.

Finally, the mechanism by which reward and fees are claimed is by inserting a transaction in the block, in the first position, that sends the total amount to the miner's public address. The network validates it with all the rest.


BTC-Wallet from "Airgap.it" asks for Receipient-Adress-Prefix "1ABC". What format is that? I am trying to set up a Cold and a Hot Wallet.

Hot Wallet = "Exodus" on my everyday mobile phone. Adressformat is Bech32 with Prefix "bc1" or legacy-prefix from format "P2PKH" then adresses start with "1".
Cold Wallet = "Airgap Vault" on offline Device plus "Airgap Wallet" on everyday mobile phone. Both from "airgap.it".
I tried to send a test amount from the Cold-Wallet to the Hot-Wallet, but Cold-Wallet "Airgap Wallet" only accepts BTC-Adresses starting with Prefix "1ABC". How do I obtain this adress from my "hot" Exodus-Wallet and how is this adress-standard/-format called?

I could only find BTC-Cash-related entries in Google, but BTC-Cash-Adresses start with "q" or "bitcoincash:q". Also the wallet software would have to be very misleading, because BTC-Cash is not even supposed to be supported. I can't even imagine that, so I guess I am really dumb right now. Any help is very much appreciated, thank you!

The Prefix "1abc" is part prefix "1" and part wildcard "abc". so it asked for the P2PKH-format all along. there is NO secret third format with a fixed prefix that is always "1abc".

I reported the issue of the QR-code accepting BECH32-format with the developers. using the right address everything works like a charm and is a very nice alternative to a hardware-wallet if you have a old smartphone to spare.


How do I use CHECKLOCKTIMEVERIFY to prevent a transaction being spent before 100 blocks? I'm creating a Bitcoin based lottery application. An entry into the lottery is a transaction spending 1BTC.

If someone guesses some arbitrary value correctly, they can claim the entry and send this 1 Btc to an address they control.

I wish to have functionality as follows: if no one guesses the correct number after 100 blocks, then I can claim it as the operator of the lottery.

Can I do this via:
Entry transaction
  IF
     CHECKLOCKTIMEVERIFY
    payable to a correct guess
  ELSE
    Payable to my operator public key
  END

Then when I try to spend this entry transaction, I set the nLockTime to the current block.

Throughout I set all the sequence numbers to 0xFFF... So that the transactions are mined.

Yes, you can use OP_CHECKLOCKTIMEVERIFY to make a transaction output unspendable for a certain number of blocks.  However, you can't use it directly in an if-else block as you have described.  When OP_CHECKLOCKTIMEVERIFY executes, it doesn't place a result value on the stack.  It either throws an error (invalidating the transaction attempting to spend the output), or continues executing as if nothing happened.

To create a transaction as you describe, you will need to use multisig, such that one party can spend the transaction as soon as the correct number is provided (input to a hash function, I presume?), OR you can spend it back to yourself after the lock time expires.  This is basically how the lightning network works.  You should check out Lightning Networks Part II: Hashed Timelock Contracts (HTLCs)


BIP44 generation tool has Account xpubkey & bip32 xpubkey, what's the difference? recently i played with the tool generation of HD address on https://iancoleman.github.io/bip39/. after input mnemonic code and check the derivation path, why is there Account Extended Public Key & BIP32 Extended Public Key? What's the difference between 2? Can i use Account xpubkey to generate the same address as bip32 xpubkey?

please advice,

BIP 44 just specifies a structure for organising keys using BIP 32 style generation. The tree structure is this:

 m / purpose' / coin_type' / account' / change / address_index

Generally, m refers to your BIP 32 master private key generated from the mnemonic in BIP 39. But all children extended keys are also extended parents if you want them to be - all can have children. So Account extended public key will just be the extended public key generated as a child of the coin_type extended key from the hierarchy above.


GUIminer, unnable to connect to pool I'm getting a connection error, or just a "connecting..." message that doesn't stop. though the connection on my end seems to be fine. This is a printout of my console. If there is already a fix for this, would someone please direct me to it? This is a screenshot of my console. I'm connecting to slush's pool if that helps.
http://i.imgur.com/vcdAY9k.png

use the GUIMiner-scrypt instead .... I could not get the guiminer to work.


Can I use Bitcoins in a game? In many games, such as Lord Of Ultima, players can purchase a special type of currency with real money, as opposed to earning it in a game. This currency is then used to purchase premium items and services, which normally are unavailable to the players otherwise.

I am wondering whether it is feasible to use Bitcoins as a form of currency in a game? Are there any games out there that have already implemented that?

There is a game that uses Bitcoin as its currency called Dragon's Tale, not to mention numerous online casinos. In general, using Bitcoins in games is quite feasible, just requires a willing developer.

In many cases it can even be preferable, as one could imagine making a simple micropayment in an instant provided they need to purchase something in a game, rather than transfer the money in bulk and later not using all of it without the chance of getting the rest back.

Moreover, similar to upcoming Diablo 3 Auction House Marketplace, or already existing market in Second Life, one can imagine the players earning Bitcoins in such games from other people and later withdrawing their earnings with an ease.


Are two factor authentication enabled wallets at mtgox or blockchain.info safe even with weak passwords? Are two factor authentication (2FA) enabled wallets at mtgox/blockchain.info safe even with weak passwords? What are the risks here, except losing one's mobile phone?

A Google authenticator is used for the 2FA, the Google account itself is also 2FA enabled.

Blockchain.info does not use 2FA for encrypting the wallet and instead uses it only for limiting access via the web interface.  If your system is compromised with a keylogger, and the attacker has access to the blockchain.info wallet (e.g., the backups sent via e-mail), for instance, then the attacker can perform a replay attack and spend the funds from the wallet.

Also, if your system is compromised the attacker might be able to obtain the password and OTP code and use that for login by simply being first to login with those credentials.  Mt. Gox implemented 2FA correctly, by requiring the OTP for each withdrawal request so that is less of an issue with them.  Not all exchanges and EWallets with 2FA require that.

So there still is better protection when when accessing your account only from a secure system, but those who live a little more dangerously can probably still keep their funds safe thanks to 2FA.

Also, the 2FA device should always be separate from the device where the wallet is being accessed from.  So accessing Blockchain for Android from a mobile in which the 2FA client is also running is not a secure configuration.


Need sample compressed and uncompressed Public / Private key pairs for BigInteger I am testing a different version of BigInteger that has different endian formats and the native implementation likes to add or remove zeros from the final output.

Can anyone provide some sample public and private keys  (not just addresses)

Compressed
Not compressed
Leading one or more zero (some implementations of BigInt mess with this) for pub and priv
Trailing one or more zero (some implementations of BigInt mess with this) for pub and private
Anything that might look "negative" when BigInteger converts an array into it's two's compliment version

A compressed version of the above would be greatly appreciated... leaving me with 8 tests.

Anyone with data that might be interpreted as a twos compliment for BigInteger, exposing bugs in the code, would be appreciated as well.

I'm doing this so the ported version of the QT client will prevent me from creating unspendable addresses.

Whomever posts the test vectors for this will likely also assist developers on other platforms (Javascript, Java, C#, etc) and their native versions of BigInteger

It's nice to see my work copied verbatim in appendix A "Blockchain: Blueprint for a new Economy" by O'Rielly... but they missed a few details. Someone from there should reach out to me

After generating 16 million keys,  comparing my results with the QT client over RPC, I've come up with the following test keys for compressed public keys:

Default

Test:
Private as hex 79186670301299046436858412936420417076660923359050732094116068951337164773779
Public address 1EE8rpFCSSaBmG19sLdgQLEWuDaiYVFT9J
Message: test123
Signature: IGP5aqo6+oKzUF4/Pq6dfksKxlcy1CehKv6aKA6GUMGOf9Jd62UWRMNwN3KXnOHi0PdBekVBlr23EKa7sGDgTtI=

___ debug data for above ___
Is public key compressed:True
Public key as hex  03A3D70082D23B32B26890EA01F6AF628F8D424DF3B025802D32FC3F2D5EF4D083
Message Hash: 9C20FD5DCB51A6DBD21CFC172CBFFA7F59DE199ED53D567EC1CAF49A4E6F19BC
Signature R (may need alignment to 32 bits): 63F96AAA3AFA82B3505E3F3EAE9D7E4B0AC65732D427A12AFE9A280E8650C18E
Signature S (may need alignment to 32 bits): 7FD25DEB651644C3703772979CE1E2D0F7417A454196BDB710A6BBB060E04ED2
Signature Noncanonical: IGP5aqo6+oKzUF4/Pq6dfksKxlcy1CehKv6aKA6GUMGOAX/SXetlFkTDcDdyl5zh4s+x8Fcr8N9d8tB5Gj0xFpA=
S value noncanonical: 173607507099637907991191162345204054084248845170247593505487421300829521874963
Key recovery value: 1

Canonical formatting of S

Test:   SValueReducedToHalf
Private as hex 104418396969965692461517188368945301704108638856440619106351414329473623554759
Public address 18m1EvVCeFsupQSsdZ3rTPGjL6j9hQpoj5
Message: test123
Signature: H5TIq/7pLqy2M9lUGUa9i1kxML4EmWN+MwiyijquyMhcbTuAiOjffxRz7xayj4qaAig8Y1OZFn/aqWDkM5GXVqQ=

___ debug data for above ___
Is public key compressed:True
Public key as hex  0228E33569266CB496AC2808C68D217B542A1E769CA62F2144BEB4D602BDB28014
Message Hash: 9C20FD5DCB51A6DBD21CFC172CBFFA7F59DE199ED53D567EC1CAF49A4E6F19BC
Signature R (may need alignment to 32 bits): 0094C8ABFEE92EACB633D9541946BD8B593130BE0499637E3308B28A3AAEC8C85C
Signature S (may need alignment to 32 bits): 0092C47F77172080EB8C10E94D707565FC927279931632206116717A593E9EEA9D
Signature Noncanonical: H5TIq/7pLqy2M9lUGUa9i1kxML4EmWN+MwiyijquyMhcksR/dxcggOuMEOlNcHVl/JJyeZMWMiBhFnF6WT6e6p0=
S value noncanonical: 66384857650308705735089161182204613834546990056514581767424835152008945003165
Key recovery value: 0

Canonical formatting of S, R is "short"

Test:   SValueReducedToHalf, RHasZeros_1
Private as hex 26841850108733854068844877092845755783895918659584173218404826715393242032133
Public address 1FRUjbVX9bbctP6y7Z7jmBufdW6KyoSvGw
Message: test123
Signature: HxypbxPc+0rICT/x6AWkGoQjE/rE7UtCLy0eS0uGn4WFGlrLkJxJDgJcIgwGDH6MdIiWL/E+cvPv2dqeHw5zCwA=

___ debug data for above ___
Is public key compressed:True
Public key as hex  03F87C4C73949612279398F2634B3D4D3DDC60EBD1D5A7F52133B9A08625DCFBC1
Message Hash: 9C20FD5DCB51A6DBD21CFC172CBFFA7F59DE199ED53D567EC1CAF49A4E6F19BC
Signature R (may need alignment to 32 bits): 1CA96F13DCFB4AC8093FF1E805A41A842313FAC4ED4B422F2D1E4B4B869F8585
Signature S (may need alignment to 32 bits): 00E5A5346F63B6F1FDA3DDF3F9F381738A3218ACF570D5AC4BE5F7C06DC1C33641
Signature Noncanonical: HxypbxPc+0rICT/x6AWkGoQjE/rE7UtCLy0eS0uGn4WF5aU0b2O28f2j3fP584FzijIYrPVw1axL5ffAbcHDNkE=
S value noncanonical: 103871533985121725711649129661651907020330641311974510999031854527404235241025
Key recovery value: 0

Canonical formatting of S, has 4 zeroes

Test:   SValueReducedToHalf, SLessThan32_4
Private as hex 86520705923672484424543748367654505450021158466896423122495789612346391207140
Public address 1NQvJ2X1KVwuYPQRdeiDbFanLod72UPEjg
Message: test123
Signature: IAD0xd4DofuKthgkVUfH8JJF/zeKeGL9b4gdYCb6PxhVcHd4KiX7lFAN2Kun9wUC21T0Z0bPnrsTlAIHWN03P20=

___ debug data for above ___
Is public key compressed:True
Public key as hex  0357B8B24DF08BEFDBEE910A3D95034FEAC1C54CDC100662FF394EB72BB23FE003
Message Hash: 9C20FD5DCB51A6DBD21CFC172CBFFA7F59DE199ED53D567EC1CAF49A4E6F19BC
Signature R (may need alignment to 32 bits): 00F4C5DE03A1FB8AB618245547C7F09245FF378A7862FD6F881D6026FA3F1855
Signature S (may need alignment to 32 bits): 008F8887D5DA046BAFF227545808FAFD2365BA759FDFA9E5282BD05733F2FF01D4
Signature Noncanonical: IAD0xd4DofuKthgkVUfH8JJF/zeKeGL9b4gdYCb6PxhVj4iH1doEa6/yJ1RYCPr9I2W6dZ/fqeUoK9BXM/L/AdQ=
S value noncanonical: 64921966049402308770510180383138525221043468997005421168542368100274768118228
Key recovery value: 1

R and S had leading zeroes (twos compliment confusion possible)

Test:   RGreaterThan32, SGreaterThan32
Private as hex 49271812223912890183372000738074590620901220493616955749035660411043423358313
Public address 1EAGUmQG2evn1ujibn1meucRt5URGr7TKS
Message: test123
Signature: H1z4EjRWIcxYLu82qXDKVwvX74EGr5Zc8hnkL8NEO9fRAGOALDrFakMTq+gVoF+qf+8jgZSI8Lomvi1sSx8Zy3Y=

___ debug data for above ___
Is public key compressed:True
Public key as hex  02A099AB8D17E11D044A495812D0619A833C03961ED0B090A06C2C81AB2CFB8C4C
Message Hash: 9C20FD5DCB51A6DBD21CFC172CBFFA7F59DE199ED53D567EC1CAF49A4E6F19BC
Signature R (may need alignment to 32 bits): 5CF812345621CC582EEF36A970CA570BD7EF8106AF965CF219E42FC3443BD7D1
Signature S (may need alignment to 32 bits): 63802C3AC56A4313ABE815A05FAA7FEF23819488F0BA26BE2D6C4B1F19CB76
Signature Noncanonical: H1z4EjRWIcxYLu82qXDKVwvX74EGr5Zc8hnkL8NEO9fRAQBjgCw6xWpDE6voFaBfqn6p0l57ODlaYn3/ytfvUAw=
S value noncanonical: 115967891712688622791697912048921273624697397818356039879425919349736894106807
Key recovery value: 0

**Canonical formatting of S,  S has more than 32 zeros, R is "short" **

Test:   SValueReducedToHalf, SGreaterThan32, RHasZeros_1
Private as hex 4777466271014663949802819381765970429152817494201827532502916324303349007402
Public address 1Ea6tSV9wrofLkfqepnbLuwDKSMgUNd8k8
Message: test123
Signature: IKJKgEi+x77OgaW0fr884GZrSh+/0cuP+TpY7EUm00EMXODggkAxLPSGfXSNWEjP5jNcSGJjjsxO1SqnNxMnowA=

___ debug data for above ___
Is public key compressed:True
Public key as hex  03AB63D0D115D9C8C53A60989D1D25EFB5FB508E1F5BEEFD7D2487D12289751E71
Message Hash: 9C20FD5DCB51A6DBD21CFC172CBFFA7F59DE199ED53D567EC1CAF49A4E6F19BC
Signature R (may need alignment to 32 bits): 00A24A8048BEC7BECE81A5B47EBF3CE0666B4A1FBFD1CB8FF93A58EC4526D3410C
Signature S (may need alignment to 32 bits): 00A31F1F7DBFCED30B79828B72A7B73018875294844BB9D3ECEAA7B755BD0E9E41
Signature Noncanonical: IKJKgEi+x77OgaW0fr884GZrSh+/0cuP+TpY7EUm00EMox8ffb/O0wt5gotyp7cwGIdSlIRLudPs6qe3Vb0OnkE=
S value noncanonical: 73781983922410365449727642718035935024524267847686688743716438700218775412289
Key recovery value: 1

**Canonical formatting of S, R and S look like two's compliment **

Test:   SValueReducedToHalf, RGreaterThan32, SGreaterThan32
Private as hex 30773778924833153278326239976267448371250861545391810261071816760957004408309
Public address 13a8JzTy6igoA7QfQyZVsDCpQvXY4eBj9i
Message: test123
Signature: H4jdYEux4k1zQaXby1OXSfdi6I7XgdkngA1F/E1rfcSHANqDQxCb3cAxnVG88p8jxyoJKPZh8y/XdYLwPhgmEMM=

___ debug data for above ___
Is public key compressed:True
Public key as hex  03C70FDC0743E19CCBF923FE5454FD6C63943E65C9BD8C1FE0847E3C365CFF8971
Message Hash: 9C20FD5DCB51A6DBD21CFC172CBFFA7F59DE199ED53D567EC1CAF49A4E6F19BC
Signature R (may need alignment to 32 bits): 0088DD604BB1E24D7341A5DBCB539749F762E88ED781D927800D45FC4D6B7DC487
Signature S (may need alignment to 32 bits): 00FF257CBCEF64223FCE62AE430D60DC3790A5B3F04D5570644A4F6E4EB810307E
Signature Noncanonical: H4jdYEux4k1zQaXby1OXSfdi6I7XgdkngA1F/E1rfcSH/yV8vO9kIj/OYq5DDWDcN5Cls/BNVXBkSk9uTrgQMH4=
S value noncanonical: 115406010640357535068751835107045199953964590515092451110614826883136174960766
Key recovery value: 0

R is short

Test:   RHasZeros_1
Private as hex 61103943668025454604648533394738966390945260249802700793400423519535802730901
Public address 19AnGbm4ZxhcUQ18Ttcpxd5AjQTtk7TfUy
Message: test123
Signature: ICLRZEqnmnOe21ZmOiKKxIpgN3F8gyxkHRYuXVJucXQAYv2FSwhrBU7djsZcDx0hSy5ilsRMBzBBIsuqkxQBEb4=

___ debug data for above ___
Is public key compressed:True
Public key as hex  032275700C1D9D1312266E4A0601DF98906C1458ACE8410FD8B6B6748FE9E65F3F
Message Hash: 9C20FD5DCB51A6DBD21CFC172CBFFA7F59DE199ED53D567EC1CAF49A4E6F19BC
Signature R (may need alignment to 32 bits): 22D1644AA79A739EDB56663A228AC48A6037717C832C641D162E5D526E717400
Signature S (may need alignment to 32 bits): 62FD854B086B054EDD8EC65C0F1D214B2E6296C44C07304122CBAA93140111BE
Signature Noncanonical: ICLRZEqnmnOe21ZmOiKKxIpgN3F8gyxkHRYuXVJucXQAAWL9hUsIawVO3Y7GXA8dIUnpEXOq+0/QfOKeCR/kN1I=
S value noncanonical: 160566680661011880010816576153459883182868735970539852654309056175300236759807
Key recovery value: 1

**Canonical formatting of S, twos' compliment formatting of R, S has 4 zeroes **

Test:   SValueReducedToHalf, RGreaterThan32, SLessThan32_4
Private as hex 73831678241472856909648189538766342154493645679368128703486299335910825024625
Public address 1MduqSPRnVWjYKNw9XyxDrTd4ScDGeB9h7
Message: test123
Signature: IADsZCQ4jSdVXJXFLMVmdvTL9Tj/P/ajJxOfbcsDUkqPANIxZZBd0aY4dqbglYBWzltXfGgVzRb6QSlDTps9bc0=

___ debug data for above ___
Is public key compressed:True
Public key as hex  03EC5FD98EA4AC0B31E9F416CE90C91297AB7DA551E413206DAD423E6EE61233AB
Message Hash: 9C20FD5DCB51A6DBD21CFC172CBFFA7F59DE199ED53D567EC1CAF49A4E6F19BC
Signature R (may need alignment to 32 bits): 00EC6424388D27555C95C52CC56676F4CBF538FF3FF6A327139F6DCB03524A8F
Signature S (may need alignment to 32 bits): 00D23165905DD1A63876A6E0958056CE5B577C6815CD16FA4129434E9B3D6DCD
Signature Noncanonical: IADsZCQ4jSdVXJXFLMVmdvTL9Tj/P/ajJxOfbcsDUkqPAQDSMWWQXdGmOHam4JWAVs0WBllOxRW3NgD7odtrc68=
S value noncanonical: 116163468044648827127288252771819760996859021381472042992761249252296984080142
Key recovery value: 1

Canonical formatting of S, S and R has twos compliment zero appended

Test:   SValueReducedToHalf, RGreaterThan32, SGreaterThan32, RHasZeros_1
Private as hex 39596642972382804109878550299664212933475275372831393917296101877715292328542
Public address 1KTMQ9PR7BTV7wofHW74nekbsDi4GYAEzN
Message: test123
Signature: IC7iMUeYPL9SuwIfs2BDoUO3cuMUVFxegg50A5HkKLeAAHyXhyVmnwgH/EzkCpITKkkn5PTAoZ6ZZv8/4PR6bAA=

___ debug data for above ___
Is public key compressed:True
Public key as hex  02218C61C3BB82864EF5646985C0BD1412C9236016D5ECFE3BB99ABDCCE39523DF
Message Hash: 9C20FD5DCB51A6DBD21CFC172CBFFA7F59DE199ED53D567EC1CAF49A4E6F19BC
Signature R (may need alignment to 32 bits): 2EE23147983CBF52BB021FB36043A143B772E314545C5E820E740391E428B780
Signature S (may need alignment to 32 bits): 00FF836878DA9960F7F803B31BF56DECD47186F7F1EEA701A258D31EABDBBBD541
Signature Noncanonical: IC7iMUeYPL9SuwIfs2BDoUO3cuMUVFxegg50A5HkKLeA/4NoeNqZYPf4A7Mb9W3s1HGG9/HupwGiWNMeq9u71UE=
S value noncanonical: 115571954394053739614724418392364175348093401631755466105511097999314105652545
Key recovery value: 1

Canonical formatting of S, s is short with 4 zeroes, r is short by 1 zero

Test:   SValueReducedToHalf, SGreaterThan32, SLessThan32_4, RHasZeros_1
Private as hex 27286465058545275041328003731375053608759800669744477817022805727071484951337
Public address 1KoZDczy6zyaRgHGVGC7pDYHVfsVo4Zo29
Message: test123
Signature: IADffy/L1F2PVhegdf0t4uFghkMH27nBSgOqK0Ikks+lUP33OvEckmcN1g1qfYRhE713I/KRT3DFJ1pP4xzwuQA=

___ debug data for above ___
Is public key compressed:True
Public key as hex  02444327032C13B19080452CED31222B90FB4F391A9E7FD8E1B01B855A759F2F4A
Message Hash: 9C20FD5DCB51A6DBD21CFC172CBFFA7F59DE199ED53D567EC1CAF49A4E6F19BC
Signature R (may need alignment to 32 bits): 00DF7F2FCBD45D8F5617A075FD2DE2E160864307DBB9C14A03AA2B422492CFA5
Signature S (may need alignment to 32 bits): 00AF0208C50EE36D98F229F295827B9EEAFD37B8F41DF92F7698780EA9B3458841
Signature Noncanonical: IADffy/L1F2PVhegdf0t4uFghkMH27nBSgOqK0Ikks+lrwIIxQ7jbZjyKfKVgnue6v03uPQd+S92mHgOqbNFiEE=
S value noncanonical: 79158342722849373070469141236626058792498908884517253652334720352725196179521
Key recovery value: 1

Canonical formatting of S

Test:   SValueReducedToHalf, RGreaterThan32, RHasZeros_1
Private as hex 108826293911840673209374265929265828788892543057636454753466906283594659795938
Public address 1Ctimb69eWuR7treaygbPcASkoqUwPGPn1
Message: test123
Signature: ILFG8FtncxYlOIC3ts7r3YI86OWT2TMD108MAokN++YjUaUPLxi1hwa+4BYf6zJ/NYIuGWG+sfK7zv8GzvONAAA=

___ debug data for above ___
Is public key compressed:True
Public key as hex  0234F964166A1A393348D7789FB3792E189C5964A0438952DB4889EA07BE819F55
Message Hash: 9C20FD5DCB51A6DBD21CFC172CBFFA7F59DE199ED53D567EC1CAF49A4E6F19BC
Signature R (may need alignment to 32 bits): 00B146F05B677316253880B7B6CEEBDD823CE8E593D93303D74F0C02890DFBE623
Signature S (may need alignment to 32 bits): 00AE5AF0D0E74A78F9411FE9E014CD80C93880C384F096AD7FF0D357BDDCA94141
Signature Noncanonical: ILFG8FtncxYlOIC3ts7r3YI86OWT2TMD108MAokN++Yjrlrw0OdKePlBH+ngFM2AyTiAw4Twlq1/8NNXvdypQUE=
S value noncanonical: 78863113940468318276170609985176761212205417632686831042339146260770630222145
Key recovery value: 1

R has 1 zero, R and S greater than 32 bytes

Test:   RGreaterThan32, SGreaterThan32, RHasZeros_1
Private as hex 35662537539778787177688141198176047348648692780898797308846230271410478493419
Public address 1CiLSHH4CXzgPkmbazysvEX8ykb8vU2feB
Message: test123
Signature: H3UW7wX01OMT7njU/f/PQtD+hCNPoIy0rALf0PEnZQYAAJYuisnyAVkVIC7qpGcSugPKnf8LHdCQLjtt60gczRY=

___ debug data for above ___
Is public key compressed:True
Public key as hex  025125B47BE00D50A535B5C9C09E86102BF182D9FE2F6F0AB4A8AC4335149C1C74
Message Hash: 9C20FD5DCB51A6DBD21CFC172CBFFA7F59DE199ED53D567EC1CAF49A4E6F19BC
Signature R (may need alignment to 32 bits): 7516EF05F4D4E313EE78D4FDFFCF42D0FE84234FA08CB4AC02DFD0F127650600
Signature S (may need alignment to 32 bits): 00962E8AC9F2015915202EEAA46712BA03CA9DFF0B1DD0902E3B6DEB481CCD16
Signature Noncanonical: H3UW7wX01OMT7njU/f/PQtD+hCNPoIy0rALf0PEnZQYAAQCWLorJ8gFZFSAu6qRnEri+eXrlumZwy+4NzHgYUw4=
S value noncanonical: 116057437519104871322947020543239762978159794193905342046496826541933056953943
Key recovery value: 0

S is not changed, R and S are large

Test:   RGreaterThan32, SGreaterThan32, SLessThan32_4
Private as hex 23984934282846201135004307705980641080962872643161302353166056218330425914143
Public address 1BcN5pYGUUChE3dk5CCiQeULESZzFYGHPC
Message: test123
Signature: IOoKQ4fmynIUV5RxzPlgU3TZr6hyyH9wXqjqU1OFagAARsDeWu8rbvlpk8MQrs+ad33BFRl3hTU03UYHtm3Kefs=

___ debug data for above ___
Is public key compressed:True
Public key as hex  02260CA828D9D5A49C6C71B52A7C54CF5FEAF2F8599C07C96FDA238072A162364F
Message Hash: 9C20FD5DCB51A6DBD21CFC172CBFFA7F59DE199ED53D567EC1CAF49A4E6F19BC
Signature R (may need alignment to 32 bits): 00EA0A4387E6CA7214579471CCF9605374D9AFA872C87F705EA8EA5353856A0000
Signature S (may need alignment to 32 bits): 46C0DE5AEF2B6EF96993C310AECF9A777DC1151977853534DD4607B66DCA79FB
Signature Noncanonical: IOoKQ4fmynIUV5RxzPlgU3TZr6hyyH9wXqjqU1OFagAAAUbA3lrvK275aZPDEK7PmnY4b/IAJs3VcJ0YZkM+ALs=
S value noncanonical: 147794757913853985161250377857925980185202154991282544344677638464716884982588
Key recovery value: 1

Rare signature format

Test:   SGreaterThan32, SLessThan32_4
Private as hex 19667464441218362309299491158177437155557792575544103707902999231594717797844
Public address 19cjvNftNKJBMcnuzHkcYrRQarjN7JMzkP
Message: test123
Signature: HwAAXzUsb6h+f1uwtl34AY6mZAhuUtzUhZwA85CjmoPfHs6AsOwzCCC3/1wjDws2JaUD0wesu1CwN9e/sFm/HmM=

___ debug data for above ___
Is public key compressed:True
Public key as hex  0372400B013BF5240116E431C1EADC00B9C2BC8A3DFFB669FB91987519ED0A20D9
Message Hash: 9C20FD5DCB51A6DBD21CFC172CBFFA7F59DE199ED53D567EC1CAF49A4E6F19BC
Signature R (may need alignment to 32 bits): 5F352C6FA87E7F5BB0B65DF8018EA664086E52DCD4859C00F390A39A83DF
Signature S (may need alignment to 32 bits): 1ECE80B0EC330820B7FF5C230F0B3625A503D307ACBB50B037D7BFB059BF1E63
Signature Noncanonical: HwAAXzUsb6h+f1uwtl34AY6mZAhuUtzUhZwA85CjmoPfAR7OgLDsMwggt/9cIw8LNiRfsq/uXAPw6/eqHj0p9V8=
S value noncanonical: 129726333383516230935110821215596586503825072828858936394280121229644796354468
Key recovery value: 0

Very rare signature result

Test:   RGreaterThan32, RHasZeros_1
Private as hex 77359564092138606367423909782286964438584967790833203478204963256314910737690
Public address 1FAzeMcAsH16Vrib9AfRT569442nAxAyWs
Message: test123
Signature: ID/UoYIc6He5C1/UOEhNPj/N4r2OIqP2Zdku0ZtXv4MecyHvpOanKO2Jjy2gO+lMSHL8DJuj/WQhzHg9g3VqAAA=

___ debug data for above ___
Is public key compressed:True
Public key as hex  020070B083903DC8EDE561B2990461DC70938898C19346D7F2FEF866629B3B0CB1
Message Hash: 9C20FD5DCB51A6DBD21CFC172CBFFA7F59DE199ED53D567EC1CAF49A4E6F19BC
Signature R (may need alignment to 32 bits): 3FD4A1821CE877B90B5FD438484D3E3FCDE2BD8E22A3F665D92ED19B57BF831E
Signature S (may need alignment to 32 bits): 7321EFA4E6A728ED898F2DA03BE94C4872FC0C9BA3FD6421CC783D83756A0000
Signature Noncanonical: ID/UoYIc6He5C1/UOEhNPj/N4r2OIqP2Zdku0ZtXv4MeAXMh76TmpyjtiY8toDvpTEctqumCU0YEXYxKnBBFoEE=
S value noncanonical: 167868026740628245794433774394316657246741244361239390199489610006892252250433
Key recovery value: 1


generating master public key from address I want to know is it possible to generate master public key from an address? if not, what is needed to generate xpub key?

It's not possible.
An address is created though a hash160 (RIPEMD160) which is a one-way function.

Creating a xpub key (bip32 extended public key) requires the according pubkey and the chaincode.
Alternatively you can calculate an xpub from an xpriv (extended private key).


Bitcoind transaction data verification I find myself a victim of either my poor googling skills, or the lack of detail in the bitcoind docs.

I have a walletNotify script running on a full node that whenever a transaction to / from my wallet arrives, performs a getRawTransaction(txid, 1) on the passed transaction ID.

Now, if any of my addresses appears inside the vout array, I know that that transaction is a deposit. What I'd like to verify, is that I can indeed spend those bitcoins.

I read about non-standard transactions (specifically, unspendable transactions) and kinda panicked, so my questions are:

Does bitcoind perform any sort of verification on the "spendability" of the arriving bitcoins?
Is there any easy way of verifying this myself? is there any not-necessarily-easy way?

Thank you very much in advance!

Bitcoind only scrapes the blockchain for a few transaction types: Pay-to-pubkey, pay-to-pubkey-hash, and pay-to-script-hash. If you check https://en.bitcoin.it/wiki/Script you'll see many more script operators that can be used, so the protocol isn't just those three script types.

Since the scripts have a very definite form, your client will look for the cases where the payments are to address YOU own instead of others.

OP_DUP OP_HASH160 <20 byte hash> OP_EQUALVERIFY OP_CHECKSIG

If bitcoins were sent to this script but with other script operations at the end, your node wouldn't report them as spendable.

Verifying this yourself is easy if you already have a working and completely correct script interpreter. You could look for non-standard outputs, since they do get mined on occasion.

However, reproducing bitcoind's script interpreter, with bug-for-bug compatibility is tricky. Certain clients have been forked from the blockchain by not following every quirk in the reference implementation.

But if you have this, you could write your own node if you liked. Listen for new blocks from the network, or just from bitcoind. Use your script interpreter to ensure that transactions in new blocks are valid, are correctly signed, and so on. Once you're confident the blockchain you end up with is correct, you can parse for the cases that bitcoind ignores and spend them if you can.


cgminer api-allow using dns instead of IP address I am wondering why cgminer is not accepting dns instead of IP addresses

I need to allow a server by it's dns...is it possible?

cgminer accepts domain names for me, It's all I have used, never have I used an IP. I would check and make sure your URI is correct if DNS is working for you in your web browser and such.

Here is an example of a URI for stratum
stratum+tcp://[serveraddress/domain]:[port]

The cgminer command would be:
./cgminer -o stratum+tcp://[serveraddress/domain]:[port] -u [username] -p [password]

If you want to test it on another pool, try this command below and try mining a couple shares on Eligius, there's no registration, just use your btc address as username and "x" for password
cgminer -o stratum+tcp://stratum.mining.eligius.st:3334 -u YourBitcoinAddress -p x

Maybe you could copy and paste some of the error messages you were getting to help further diagnose this?


Are there any instruments that I can use to trade bitcoin volatility? In the stock market, there are several financial products that allow an individual investor to trade/hedge daily volatility? As an example, there is the VIIX. Are there any such financial products available in the bitcoin community? IF one wanted bitcoin exposure for a day or two, due to a major event, is the only way to buy bitcoins themselves?

A VIX index can easily be reconstructed.

Trading that index should be somewhat achievable with a multi-signature transaction, depending upon the limitations of the implementation.

If it is completely private without any regard for transparency at full counterparty trust, a binary option can easily be constructed.  If the BTCVIX is >= X at expiration, pay 100% to party A else pay to party B.  The possible price is (0,100)%, rationally speaking where one party "buys" at P% and the other party "sells" at 100-P%.  In reality, both parties deposit the funds into the transaction, and the winner takes all.

The problems involved with multisigs have been extensively discussed here and elsewhere, but layers can be added to the transaction to ensure smooth functioning such as posting the text of the terms of the transaction in the transaction, both parties posting margin which is returned upon exercision, a 3rd party that cosigns with one party to free the funds, etc.

The complexity thus costs skyrockets with vanilla options because of having to recreate a margin mechanism.


How will node handle two branches of same length? Let's say node is on a chain of length N+4, and receives a block from the chain of the same length N+4, but diverged by 4 blocks.

Is the information of the length and accumulated difficulty broadcasted along with mined block?
How will the recipient node handle this situation?

This phenomenon is known as a chain-split (i.e. the bitcoin network can no longer agree which chain is the canonical chain).

Nodes will be network-partitioned until one chain becomes dominant.

Therefore:

  Is the information of the length and accumulated difficulty broadcasted along with mined block?

No.

  How will the recipient node handle this situation?

Nodes will handle equal branches on a First Come, First Serve basis, so there will be a network partition where some nodes agree chain A with length N+4 is the canonical chain and the other set of nodes will assume chain B with length N+4 is the canonical chain. In the event of such a chain-split, the race is on for miners to produce a new block-tip and broadcast it across the bitcoin network ASAP to achieve convergence. When the contention is resolved, the node can switch to longer branch and obsolete the old branch.


Does a trusted blockchain import from BitcoinCore require that chainstate and blocks be from the same source? Can the chainstate folder from a pruned node be used with the blocks folder from a full node to provide the full blockchain?  Assume that both nodes have been running for months.
If the blocks folder contains references to filenames in the chainstate folder, then the answer is no.  I know this because the filenames in chainstate on the full node are different from those on the pruned node.
If, on the other hand, the chainstate folder stands as a self-contained and full record of the UTXO Set including all pointers, filenames, and identifiers required to interpret it, then it seems that I will NOT need to import chainstate from the full node to the cloud server.
Bonus question! Out of curiosity, there's another question here that doesn't affect my situation, but it would be nice to understand why the answer is no if it's no.  If the chainstate folder on a full node got corrupted, could the chainstate folder from a pruned node be dropped in to recover?
Clarification
Both the blocks and chainstate folders reflect a block tip, and if the two folders reflect different block tips, they will be incompatible.  I should have made it explicit that the question assumes that the two different sources of folders would both be synced up to the same block height.
An interesting edge case is what happens if they are using the same block height, but the blocks they used at that height are two valid but different blocks.

It’s not self contained. The chain state contains references to the positions within the blocks which are unique to the sync state.


Transfer BTC from paper wallet without 3rd party (like blockchain.com etc) I have btc on my paper wallet and I want to transfer to some other address. How can I do this without sharing my private key to some other site like blockchain.com or electrum?

Electrum does not share your private key with its server. I would use Electrum as it's very convenient, well-maintained and it does not have large storage requirements.

Create new wallet
File > Sweep private key

Some other options are:

Bitcoin Core, which expects you to download the whole blockchain. Use the importprivkey command.
BRD wallet, which is my favorite mobile wallet that can sweep from QR.
CoPay mobile wallet also has the same feature.

Electrum and Bitcoin Core are the two most popular desktop wallets. (Core was actually designed as a full node, therefore it's not very user-friendly and not the best wallet). I'm not aware of a third desktop wallet.


Bitcoin-qt 0.8.5 upgrade I used to mine bitcoins and stopped about two years ago. At that time I was using Bitcoin-QT 0.8.5 Beta. The computer has been powered off since then, and my wallet.dat is still on that computer.

Can I still use that version, or do I have to upgrade to the latest Bitcoin Core? Besides backing up my wallet, should I do anything in preparation before I reconnect the old computer to the Internet, or before upgrading software? It says the wallet is out of sync and the block is 95 weeks behind.

Any guidance will be appreciated.

Thanks.

Can I still use that version, or do I have to upgrade to the latest Bitcoin Core?

You can still use that version. I would upgrade to the latest stable version, though.

  Besides backing up my wallet, should I do anything in preparation before I reconnect the old computer to the Internet, or before upgrading software?

No, you're good.


Block nonce predictable Is the block nonce predictable in any way? It seems to be a proof of work on his own just like the block hash, but may a miner force a mined block to have a specific nonce? (if his hashrate allows him to)

No it is not predictable. If it was that would break how block discovery works.

To explain in very broad terms: a nonce is simply an incrementing number. It is combined with a random set of as yet unconfirmed transactions and passed through a simple hashing algorithm, which turns it into a long number called a hash.

It's this answer (the hash) that has to match the test : it needs to begin with a certain number of zeros for the block of transactions to have been considered "discovered", and hence confirmed.

If it does not match this test, the nonce is incremented by and the process begins again.

It is the fact that the number and content of the transactions combined with the nonce supplies the essential unpredictability of the process, meaning that ultimately nobody can say which miner will find the block.

As this is quite a heavy computing task (to continuously try new nonces and transactions until you find the block) the nonce also serves another purpose : to help other bitcoin nodes take a shortcut to check if the block you claim you discovered is correct. They simply need to take all the transactions in the block (you have to tell them which ones) and use the nonce to arrive at the correct Hash with the right number of zeros. This by contrast is a very light computing task.


What is Best Address to use to receive multiple payments? I plan on attaching my Bitcoin Addresse to a program that will pay me multiple times. I plan on setting it and using only one. So which address is the best to use, the one that starts with BC or the number 1?
My fear is that if I put the one BTC address that keeps changing onto the program, then I may not receive multiple payments. Because the address wouldn't be good.
Share Edit Flag

All those addresses have the same structure (a public key hash) and are used the same way.
The difference is, bc1 addresses keep transaction signatures in a separate place in the blockchain than 1 addresses. That is, there's a fee discount when you spend from bc1 addresses. Without any doubts, the newer bc1 addresses are superior.
The reason 1 addresses are still in use is, legacy codebases that aren't updated for bc1 compatibility. Use bc1 whenever possible.


When using createrawtransaction, are the outputs ordered as they are presented within the request? Let say, when I set second parameter of craterawtransaction call as below

[{"demoaddress1": 0.42}, {"demoaddress2": 1.00}]

Is it possible to occur that returned transaction's output is not ordered as same? like below

[{"demoaddress2": 1.00}, {"demoaddress1": 0.42}]

I found the answer on mailing list.

https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2018-October/016441.html

  The createrawtransaction RPC will now accept an array or dictionary (kept for compatibility) for the outputs parameter. This means the order of transaction outputs can be specified by the client.

According to the description, it should be the same order.


How can I get bitcoins in Canada? I want to exchange my Canadian dollars for Bitcoins. How can this be done? If it helps I have some American cash.

These are a few options available to Canadians:

Local Bitcoins
BitBuy
CoinSquare
QuickCoin

There are also a few exchanges that accept CAD deposits larger than $100:


Safely transferring bitcoins from wallet.dat to cold storage A similar question was asked by me in 2014, but I did not find the answer satisfactory back then, and I hope some things changed since in the last two or so years.

I have a wallet.dat file with some bitcoins on it. I would like to move the bitcoins on the file to an offline paper wallet.

Is there a convenient way to make the transaction without connecting the computer with the wallet.dat file to the internet?

The problem here is that creating a transaction requires 2 things:

The utxo (unspent transaction output) set of your addresses. This is basically a record of all bitcoins sent to the addresses in your wallet.dat file.
The private keys to actually sign the transaction.

Generally, when you use an offline computer to store the private keys, you will be using an internet connected computer to do (1) and the offline computer to do (2).

Unfortunately, the bitcoin-qt client does not have the concept of watching-only wallets for (1) which would be required for you to create such a transaction. That said, there are some ways you can achieve this, but it's definitely not going to be easy.

Here are some options:

Manually grab the utxo set for your addresses using a block explorer (like blockchain.info), then manually create a transaction spending those bitcoins to the paper wallet address, then sign the transaction on your offline computer and finally broadcast the transaction online.
Download the blockchain on an online computer (about 50 gb), then transfer that over to your offline computer, then create the transaction you need on your offline computer and broadcast the transaction online.
Dump the private keys of the addresses on your offline computer and create paper wallets of the private keys that have any bitcoins in them. (This can be achieved using something like bitcoinpaperwallet.com and entering your own private key)

1 is going to require some programming experience. 2 is going to take a while to download the blockchain and transfer it. 3 is probably your easiest option depending on the number of addresses you have. The nice thing about bitcoinpaperwallet.com is that you can download the source and use it on your offline computer.


Actively display wallet funds on web page - how? I am working on a project where people can contribute to the project in Bitcoin or Ethereum. For each I need a way to be able to display an active balance in each currency's wallet. Something along the lines of this:
You have elected to contribute in BTC to the project. Thank you. Your contributions are most welcome. Please send your BTC to 123abc456def. This wallet currently contains X Bitcoins, while Y have been contributed in total, and the contributed funds have been used for the purposes listed below.
And each time the person accesses the page, X and Y will reflect that wallet at that moment. I'd really prefer not to run a full copy of the blockchain for this. I already have bitcoind running with a 4GB pruned database, and I'm hoping that will be enough.
I need to do all this in PHP. Is there a good PHP toolkit/library/whatever that I can use to call the info?
Thanks!

I already have bitcoind running with a 4GB pruned database, and I'm hoping that will be enough.

It is! By running bitcoin-core, you can pull this data in an easy, trustless way.
I would recommend creating and displaying a new address to your patrons each time a new donation is received (so as to avoid address reuse, which is detrimental to your privacy, and perhaps more importantly your patron's privacy), but otherwise you should be able to accomplish your goal without much issue. You could perhaps look at something like btcpayserver to help automate the payment flow on your website.
Here is a general workflow:

Generate a private key that will be used to receive funds, using your wallet of choice (ideally, this would be done in an offline environment)
Import the address(es) that are derived from the keys you generated in step one, into bitcoin-core. You can use bitcoin-cli importaddress to accomplish this step. Note that if the address is brand new and you know for certain it has never received funds before, then you can set rescan to false, to save a fair bit of computational time.
Use the command bitcoin-cli listunspent to return a list of UTXOs that are locked to addresses that the wallet is watching. You can optionally filter the returned info by address, see bitcoin-cli help listunspent for more info.
(Alternatively, you can use bitcoin-cli getreceivedbyaddress, which may be easier if you're re-using a single receiving address)
From there, you'll just need to parse the JSON data returned, and sum up the relevant address balance(s), to be displayed on your webpage.


In BitcoinJ, what is the purpose of Peer? In BitcoinJ, there is PeerGroup and Peer.

I understand there are differences such as one is related to a group of peers while other is related to a single peer.

When should I use which construct? Are both necessary?

From the code, it seems that we should only be interested in PeerGroup and not Peer. Then whats the purpose of Peer?

When should I use which construct?

Use PeerGroup, unless you want to connect to a specific Bitcoin node.

  From the code, it seems that we should only be interested in PeerGroup and not Peer.

PeerGroup will construct Peer objects. It's necessary, even if you don't use it directly.


Why might MultiBit not show a transaction that was sent to me? So i recently bought a bitcoin from a vendor , and i had it sent to my Address in multibit.

Now the problem is , the money isn't there and i'm not sure what to do.

He told me to get bitcoin (program) and now it's starting to download a chain. Currently it's at 95000.

I'm not sure what to do exactly , as i'm worried i may have lost my funds.

What can i do at this point? Do i wait till the Blockchain is finished or should i import my wallet from Multibit to Bitcoin?

Thanks guys!

-D

You can ask the vendor for the transaction id of the transaction where they sent the bitcoin to you. Look it up on https://blockchain.info to check it exists and is sending bitcoin to your receiving address correctly. Look at the date of the transaction. Go into MultiBit, select your wallet and choose the menu option 'Tools | Reset blockchain and transactions'. Reset from the day before the transaction. Once it has resynced to the network you should see it in MultiBit.


What is the daily transaction limit for Bitinstant? I know that bitinstant limits individual trades to 3 digits or $999 per transaction, but they make no mention in their FAQ about the size of the daily limit they impose.

I was wondering if anyone knew the exact number?

Note: I think this number may vary depending on where your sending money from, however I'm specifically interested in the limit regarding sending money from mtgox to bitstamp and vice versa.

Update: I emailed them and they confirmed that the limit is in fact $2000 per day.

Update: Since this question bitinstant has updated their service. You can now be verified by sending in a photo idea and have your limit upped to $10,000 per day.

$2000 per day. Right above their captcha-like thing, it says the limits in small text.


Can I recover my bitcoins from an Armory paper wallet even if Armory ceases to function? I just created an M-of-N offline fragmented backup paper wallet through Armory. I plan to put most of my BTC here. Before I do, I want to know that if anything should happen to Armory -- if they stop updating it, or I can't get it to run on my computer in 10 years, or there were some updates to Bitcoin-Core/the blockchain, or whatever -- will I still be able to get my coins out?

Short answer, you need Armory to open an Armory paper wallet.

I use Armory and it does not fill me with confidence. I set one up and it was taking forever to sync with the blockchain. I think it actually crashed. However I had bought some BitCoin which was in there somewhere but not showing. I made a paper backup and formatted the hard drive because the PC was playing up.

8 months later I discovered my Armory paper wallet saved as a PDF and decided to see if I could get hold of my BitCoin. The codes did not seem to be any use with any online tools I could find. It would not import into an online BitCoin wallet. I did not want to have to download the 140GB blockchain.

Anyway I did install Armory and activated the restore process for the paper wallet. It restored fine but seemed to be empty and I wondered why I had bothered saving an empty wallet that had never had transactions. The process of getting Armory properly running is like firing up a steam engine compared to turning the ignition on a car. I had to also install the normal bitcoin-qt in order to get the block chain. This took two weeks. Even then Armory was not happy. Several restarts later Armory had synchronized it's database and all the other nonsense it has to do. That took a couple of days or so.

Finally I discovered the wallet had £150 worth of coin in it! Dead chuffed.

I am good with computers but I found all of this hard work and stressful. The average person will need all this done for them. I'd put it on the same difficulty level as formatting your hard drive and re-installing the OS and all the drivers. Not the hardest thing in the world but beyond most people.

I'd say any sort of wallet you use needs to be backed up in a way that it can easily be examined in a couple on minutes rather than the 2 or three weeks it took me. It may well be down to my lack of experience but I did not get the impression that an Armory backup could be used on anything but Armory.

As a little rant I'd say Crypto has to get much easier than currently if more than just technogeeks are to use it.


How to cash out E*Trade stocks (transfer funds from US to the EU) with Bitcoins? My company has an employee stock purchase plan with etrade.com, but as I'm in Europe, I thought I could sell the stocks and save the bank transfer fees by using bitcoins.

You can sell the stocks online on NASDAQ though, but even to just place the order they ask for an bank account. So I either have to transfer to EU, paying the fee, or to an US account. As I don't have one, could I simply transfer directly to an Bitcoin exchange?

*You basically need to get money into E*Trades US back account.*

E*Trade will either want a wire transfer (from Europe) or a bank transfer from the US.
They have to be very careful about Anti Money Laundering. So  wont want to accept money directly from a Bit-Coin Exchange.

You can open a US account from your local EU bank. Your local bank will act as a proxy to make the required AML and KYC checks. You can then use bitcoin to route the money via a bitcoin exchanges to that account.

However if this is just a one off. You will probably find the costs, (fees at the EU exchange and the US one) and risk of bit-coin volatility; not worth the effort.

Remember this is a problem which existed before bitcoin. So there will be ways of getting it to work with out bitcoin.

I would contact E*trade EU and US and ask them what would be the best way to find you purchase. (E*trade EU may have an arrangement to do this simply, already)

If you are buying a large enough quantity of shares, then the wire fees etc will only amount to a few percent anyway.

That being said having a US acc would be useful for BTC stuff later on. For example if want to work remotely for a US company, and need an efficient way of transferring money regularly to you home currency personal bank account.


Problems checking BTC current balance using blockchain.info's API, am I correct here? I have a script that uses the Blockchain API to check the current balance on all my accounts one by one. When I reach a certain balance I sweep to a master account.

Am I using the correct code here as I am having errors checking the balance on accounts.

my $mech = WWW::Mechanize->new();
$mech->get("http://blockexplorer.com/q/getreceivedbyaddress/".$btcaddress);   # Check transactions on each address as per masterlist
my $getreceived = $mech->content;

$mech->get("http://blockexplorer.com/q/addressbalance/".$btcaddress);   # Check balance on of each address as per master list
my $addressbalance = $mech->content;

I can check transaction volume ok to reconcile with my accounts but I want to know the current final balance too so I can check that box also.

Just ran almost the exact same code in perl and this works:

#!/usr/bin/perl

use WWW::Mechanize;

my $btcaddress = "1Enn7YW22vBGvCWrsPQN7vDhnQCYhy221V";

my $mech = WWW::Mechanize->new();
$mech->get("http://blockexplorer.com/q/getreceivedbyaddress/" . $btcaddress);   # Check transactions on each address as per masterlist
my $getreceived = $mech->content;

print $getreceived . "\n";

$mech->get("http://blockexplorer.com/q/addressbalance/".$btcaddress);   # Check balance on of each address as per master list
my $addressbalance = $mech->content;

print $addressbalance . "\n";

Note the address is a random one I found on blockchain.info.


Found my old wallet, is it empty? So I found my old wallet from 2011, I was able to pull my address and my PrivateKey using  pywallet. I downloaded Bitcoin-QT, copied the old wallet over and reran with -scan. Since it was a new install the blockchain is still updating (not finished as of this post).

I searched my address at blockchain.info and it said that address didn't have a balance. With that being said, are my efforts at recovering the wallet pointless? I pulled the wallet off an old drive so it hasn't connected since 6/2011. The entire directory is intact and the block that was processed is still in the directory as well, about 301MB big. Not sure if that helps me at all in this case.

EDIT: Here'e the pywallet dump with critical data removed. The fact that there is nothing under tx makes me believe that I'm wasting my time.

{
"bestblock": "REDACTED",
"ckey": [],
"defaultkey": "1PSatNhRDmmR6rfK5c9XJvqwB92z7beRa",
"keys": [
    {
        "addr": "1PSatNhRDmmR6rfK5c9XJvqwB92z7beRa",
        "compressed": false,
        "hexsec": "REDACTED",
        "label": "",
        "private": "REDACTED",
        "pubkey": "REDACTED",
        "reserve": 0,
        "sec": "REDACTED",
        "secret": "REDACTED"
    }
],
"mkey": {},
"names": {
    "1PSatNhRDmmR6rfK5c9XJvqwB92z7beRa": ""
},
"pool": [],
"settings": {
    "addrProxy": "127.0.0.1:23075",
    "fGenerateBitcoins": "False",
    "fUseProxy": "False"
},
"tx": [],
"version": 32100
}

It's worth keeping in mind that your "wallet" doesn't actually contain any bitcoins. What your wallet contains is the private keys that permit you to spend coins that have been sent to any of your addresses. Which addresses have been sent coins is a matter of public record (the blockchain).

If you had previously received (or mined) coins using one or more of your addresses that are contained in that wallet, then the record of those transactions will still exist. Note that your wallet likely contains more than one address, particularly if you've used it for anything at all. The address you have shown in your question has not received any coins.

If you have copied your old wallet.dat file into a new Bitcoin client, the first thing to do would be wait for the blockchain download to complete. Your client won't know about any coins that have been sent to the addresses in your wallet without actually seeing the whole blockchain.


Bitcoin mining: block structure please note I have made this question previously (link) and I was indicated to visit here.

I am writing a program using GoLang to mine bitcoins. It's part of my college conclusion project and likely won't be used to mine bitcoins since it's not gonna be efficient enough running on regular PC's (facing a race against those huge chinese mining pools). Though it's not originally intended for real usage, my college expects nothing less than a flawless piece of software to mine real bitcoins (in parallel).

So far, I have implemented quite an extensive code, therefore, you guys may assume I already have means to achieve:

  Get pending transactions to fill my block (from https://blockchain.info)
  Build the merkle tree over the transactions data retrieved
  Calculate a sha256 of a given data input (I also understand the concepts behind such hash functions)

I have organized my questions regarding this subject below:

Question 1) I can't precisely tell how to build a block header which would be appended to the block chain. I know it has fields like the root of our merkle tree (which I already know how to calculate) or a version field, but if I was to produce a block header as an array of bytes, how would I do it ? I'd like to see how to calculate each field we need to build the header and how our byte array (representing the header) would look like.

Any dummy data may be used for this example, my focus stays on understanding the process and the data structure.

Please, note I have already read this: https://bitcoin.org/en/developer-reference#block-headers

I have studied quite a lot regarding how the bitcoin protocol works to achieve the code I currently have, but I feel extremely limited on the information I can find online regarding the internal details of the bitcoin protocol:

Question 2) Is there any online technical information you guys would consider useful for me to read regarding the headers subject and/or anything else in bitcoin? Keep in mid that I still have quite some code to write, therefore, I'm taking all the information I can get : )

As an example of how to build a block header, here's a short Python program that calculates this block header's hash:

#!/usr/bin/env python3
import urllib.request
import json
import binascii
import struct
import hashlib
ux = binascii.unhexlify
hx = lambda bin: binascii.hexlify(bin).decode('ascii')

# Load testing data in json format from blockchain.info
url = "https://blockchain.info/block-index/1114939/000000000000000003e6d3647001d1f455d585cdd3c298093902fe52ea4529ba?format=json"
block = json.loads(urllib.request.urlopen(url).read().decode('ascii'))
print("Original block hash: " + block["hash"])

version = block["ver"]
# 4 bytes, little endian
version = struct.pack("<I", version)

print("Version: " + hx(version))

prev_block = block["prev_block"]
# Reverse hash to go from display format to internal format
prev_block = ux(prev_block)[::-1]

print("prev_block: " + hx(prev_block))

merkle_root = block["mrkl_root"]
# Reverse hash to go from display format to internal format
merkle_root = ux(merkle_root)[::-1]

print("Merkle root: " + hx(merkle_root))

timestamp = block["time"]
# 4 bytes, little endian
timestamp = struct.pack("<I", timestamp)

print("Timestamp: " + hx(timestamp))

target = block["bits"]
# 4 bytes, little endian
target = struct.pack("<I", target)

print("Encoded target: ", hx(target))

nonce = block["nonce"]
# 4 bytes, little endian
nonce = struct.pack("<I", nonce)

print("Nonce: ", hx(nonce))

combined = version + prev_block + merkle_root + timestamp + target + nonce

print("Block header: " + hx(combined))

sha256 = lambda x: hashlib.sha256(x).digest()
sha256d = lambda x: sha256(sha256(x))
computed_hash = sha256d(combined)

# Reverse hash to go from internal format to display format
computed_hash = computed_hash[::-1]

print("computed_hash: " + hx(computed_hash))

Here's the output.

Original block hash: 000000000000000003e6d3647001d1f455d585cdd3c298093902fe52ea4529ba
Version: 01000030
prev_block: cd0594ebccfe15b205125165cc3b66986ed5b2a311cd88050000000000000000
Merkle root: 37022f7a3b9a2b199d53fbf1552be32c141892319966c3ef999d348d74cfb582
Timestamp: 7f476357
Encoded target:  a09b0518
Nonce:  ae5ae1c1
Block header: 01000030cd0594ebccfe15b205125165cc3b66986ed5b2a311cd8805000000000000000037022f7a3b9a2b199d53fbf1552be32c141892319966c3ef999d348d74cfb5827f476357a09b0518ae5ae1c1
computed_hash: 000000000000000003e6d3647001d1f455d585cdd3c298093902fe52ea4529ba


How to denote  script in P2SH transaction? From the bitcoin developer page, the format of the scripts used in P2SH transactions are as follows:

Pubkey script: OP_HASH160 <Hash160(redeemScript)> OP_EQUAL
Redeem script: <OP_2> <pubkey> <pubkey> <pubkey> <OP_3> OP_CHECKMULTISIG
Signature script: OP_0 <sig> <sig> <redeemScript>

When coins that were previously sent to a P2SH script are spent, and so the concatenated scriptSig and scriptPubKey scripts are executed, the OP_HASH160 is calculated on the redeem script, which is part of the scriptSig. How is it denoted in the script which parts should be included in the redeemScript? That is, how is it denoted which parts should be hashed when OP_HASH160 gets executed? Obviously it can't be the whole scriptSig, because there are signatures contained, so how does the executor know what to hash? Is there some sort of a marker OP code?

Thanks!

Short Answer
Yes, but there's more than one.
Long Answer
Bitcoin scripts are a stack-based language. The 'separators' are the Bitcoin push-data-onto-stack instructions. There are multiple instructions to push data onto the stack:

OP_0, OP_1, OP_2, ...
0x01-0x75 (Yes, that's their real name)
Push opcode bytes onto the stack
This is most common - the average piece of data in Bitcoin is shorter than 75 bytes.
PUSHDATA1 - Read 1 byte, convert to int, read that many bytes
PUSHDATA2 - Read 2 bytes, convert to int, read that many bytes
PUSHDATA4 ...
OP_1NEGATE - Pushes -1 onto the stack. Somewhat use-impaired.

There's a useful rule of thumb here, though it's not always correct. Take the first byte. Convert it to a number. Count that many bytes forward. That's a single piece of data. Repeat.
Let's work through an example. Let's find the elements of the input script for this transaction.

0100000001cce488677848b7a962f01a525d7cd78139a5be1ab75ead5b7336ba6f512d0388010000008b483045022100b65463abe3b5f79dbe9556180e60db4b6b3a6db7c43132e1e396e73023b34d2402206a412868470f8b83dc266a364b79e0a2acd0f5642181f7d0212c7ea608271a4a0141040ab8f88b419c62dba59d2e685bd3bc07367236d841abf3068b5886cd7a6c7297e721d42a41cb1bbd69ed684cc5d5c738d611d886c4493280b26637abad5603a5ffffffff0280f0fa02000000001976a914f593d5cfb3b74ac2988815cdf26c0420981385b788ac50d2a019000000001976a9148a0c13d08a36e68aa5294ca7e6f06a9946c3f1d588ac00000000

Break it up, find the input script:

version: 01000000
inputs: 01
hash: cce488677848b7a962f01a525d7cd78139a5be1ab75ead5b7336ba6f512d0388
vout: 01000000
input script len: 8b
input script: 483045022100b65463abe3b5f79dbe9556180e60db4b6b3a6db7c43132e1e396e73023b34d2402206a412868470f8b83dc266a364b79e0a2acd0f5642181f7d0212c7ea608271a4a0141040ab8f88b419c62dba59d2e685bd3bc07367236d841abf3068b5886cd7a6c7297e721d42a41cb1bbd69ed684cc5d5c738d611d886c4493280b26637abad5603a5
seq: ffffffff
outputs: 02
amount: 80f0fa0200000000
output script len: 19
output script: 76a914f593d5cfb3b74ac2988815cdf26c0420981385b788ac
amount: 50d2a01900000000
output script len: 19
output script: 76a9148a0c13d08a36e68aa5294ca7e6f06a9946c3f1d588ac
lock_time: 00000000

Now, we know the first piece of data is 0x48 (72) bytes long.

48...3045022100b65463abe3b5f79dbe9556180e60db4b6b3a6db7c43132e1e396e73023b34d2402206a412868470f8b83dc266a364b79e0a2acd0f5642181f7d0212c7ea608271a4a01...41040ab8f88b419c62dba59d2e685bd3bc07367236d841abf3068b5886cd7a6c7297e721d42a41cb1bbd69ed684cc5d5c738d611d886c4493280b26637abad5603a5

The second part is 0x41 (65) bytes long:

48...3045022100b65463abe3b5f79dbe9556180e60db4b6b3a6db7c43132e1e396e73023b34d2402206a412868470f8b83dc266a364b79e0a2acd0f5642181f7d0212c7ea608271a4a01...41...040ab8f88b419c62dba59d2e685bd3bc07367236d841abf3068b5886cd7a6c7297e721d42a41cb1bbd69ed684cc5d5c738d611d886c4493280b26637abad5603a5...


"accounts" in bitcoin-qt I've read (i.e. here https://en.bitcoin.it/wiki/Accounts_explained) about the concept of "accounts" in bitcoin.  However I can't see how to create them in bitcoin-qt.   How do I use accounts?

The accounts feature is only accessible through the RPC interface.

Also, unless you want to maintain separate balances for several users sharing the same wallet, it's unlikely you'll want to use this feature.


Recovering Blockchaininfo wallet in Electrum wallet I have my wallet in blockchaininfo and I did the backup of my 12 words (SEED).

I would like to open this wallet in Electrum, I notice that Electrum allow to recover a wallet using a SEED from another wallet (BIP39 seed).

When I type my seed and I choose the BIP39 option, I can click on next step where I have to choose the type of the addresses on my wallet: legacy (p2pkh); p2psh-segwit (p2wpkh-p2sh); native segwit (p2wpkh). I am not sure what those options mean, so I just choose the default option (native segwit). After this step, I just need to set up a password.

But when I do all these steps, I cannot see any Bitcoins in my Electrum wallet, Am I doing something wrong in the steps or is not possible recovering blockchaininfo wallet on Electrum wallet?

A mnemonic seed phrase can be used to derive an unbounded number of addresses, so you need to supply some additional information, so that the wallet software will know which addresses to derive.

This additional information includes the derivation pathway, and the address type.

There are some standards developed to specify these variables, see for example BIP 32, 39, 44, 84.

So to recover your wallet, you'll want to figure out first which derivation path the original wallet used (you should hopefully be able to find this in the software docs / help), as well as the address type used (should be self-evident, based on the address prefix).


Can one spend part of multisig address balance leaving the rest intact? Is it possible to partially spend the balance of multisig address? Let me explain what I mean.
Let's say I have a 2-of-3 multisig address with no BTC in it. Then John sends 0.5 BTC to the address and Alice later sends another 0.8 BTC to the address. Is it possible to create a redeem script to spend 1 BTC from the address and later create another one to spend 0.3 BTC?

Yes.
Funds can be spent from a multisig address in the same way as from other addresses, except that you need to have the required number of valid signatures. Each UTXO must be spent in its entirety, but you can
create a "change" output which is sent back to the same multisig address. So it effectively works like partial spending from a user's perspective.

  1 BTC from the address and later create another one to spend 0.3 BTC?

Note: You would have to account for tx fees, so slightly less than that.


Is there any risk associated with using the same passphrase on two different wallets both hosted on blockchain.info? Does blockchain.info use any salt with the passphrase?

I.e., is there any risk associated with using the same passphrase on two different wallets both hosted on blockchain.info?

edit: of course blockchain.info doesn't even keep the hash of the passphrase, as opposite to traditional login systems, so it might not make sense to use a salt in the first place... or does it?

The risk is that, if a hacker gets a hold of one, then they've gotten a hold of both. Or at least If I had one passphrase, it would be my first guess for the second.

However I dont think there is any inherent technical problem for blockchain.info.


How does a perpetual futures contract affects the actual price? I kind of understand how a perpetual futures contract works, but I don't see how it affect the price of the underlying asset, such as Bitcoin.

The majority of volume is traded via perpetual swaps, hence the majority of belief of the commodity price must lie in those perpetual swaps.  For this to be true, it would mean that the perpetual swap affects the spot price in the first place.
As the perpetual swap is a derivative of the actual commodity they share the price changes with each other by the no-arbitrage constraint as per answer here.
https://qr.ae/pNaIuG


Does blockchain work like Git? So the blockchain is a linked list structure, but the transaction chain is a DAG, that means that bitcoin works a bit like Git?

In a way, yes.  Each block contains the differences of each address and the result becomes the current UTXO set.


What do they mean by "Make sure to send enough to cover any coin transaction fees!"? When I paid a Bitcoin bill yesterday to Coinpayments.net (that's what they used), which had no problems, I spotted this text:

Make sure to send enough to cover any coin transaction fees!

What exactly do they mean by this?
I used their bitcoin: hyperlink in Bitcoin Core so I got the "pay form" filled in just as they wanted it, and again, I had zero problems. I just wonder what they could possibly be meaning by this? "Send enough"? Cover "any coin transaction fees"?

Perhaps they meant "don't subtract fees from the amount" That is, if you owe 0.1 BTC and you have 0.1 BTC in you wallet, you shouldn't pay.


Multibit transaction is not being confirmed I'm using MultiBit version 0.5.16 (is this classic?)
I am trying to get my bitcoins from Multibit to Electrum, but my transaction doesnt get confirmed. The current transaction is 1mBTC to test it first, so it's not a huge deal if I lose this 1mBTC.
I saw here something similar: Transactions seen by peers, but not confirming
The answer there is that the version is too old.

As MultiBit is no longer supported, is it even possible to update safely, to make sure my transaction will be confirmed?

Thanks in advance,

RitchBit

Move your private keys to Electrum. I had this issue with Multibit HD since last summer. I sent my bitcoin from Hive (which had just become unsupported) to multibit hd. For over a year the transaction showed up on multibit as unconfirmed. The steps I took were:

Recover the wallet words from the multibit hd wallet file
Use a tool online that converted the multibit wallet words to private keys
Added the private keys to Electrum wallet

As soon as I created the electrum wallet with the private keys it showed up as confirmed in electrum. I would completely move to another wallet altogether as multibit is deprecated, and I’ve seen this unconfirmed bug happen quite often, don’t use multibit anymore.


How to confirm that a bitcoin transaction is confirmed and success? Coming from ethereum background, I am used to the underlying ethereum tx concept where tx id is not available (returns null) until it is being mined. And after the tx is being mined it have three status which are pending, success and failure, which refers to the transaction state.

Is this the same as bitcoin as I could not see the state of transaction in bitcoin. I can't find any status of the transaction in txid of api returns (i use insight api). How could i ensure that the transaction is in pending/complete state and how can i ensure that the transaction is success or failure. I am thinking of checking 6 confirmations but I don't think that this is a "safe" and proper way as confirmation numbers varies.

To summarize
1) How do i ensure that a transaction is in pending or completed state
2) How do i ensure that a transaction is in success or failure state after completed.

What I did in ethereum to confirm the transaction is completed and success:
1) Keep looping and call the txid api every x seconds to get tx detail until it returns non-null object
2) if it returns an object, i can check the status (which is success or failure) to confirm that the transaction is success.

How it is done in bitcoin way?

Sample output of insights api tx:

{
    "txid": "",
    "version": 2,
    "locktime": 0,
    "vin": [
        ...
    ],
    "vout": [
       ...
    ],
    "blockhash": "0000000000000000000586ebc988140383fd557f9bbc09ca0ac10cff2cb0fafd",
    "blockheight": 525860,
    "confirmations": 9,
    "time": 1528080051,
    "blocktime": 1528080051,
    "valueOut": 3.27637624,
    "size": 290,
    "valueIn": 3.27737624,
    "fees": 0.001
}

Bitcoin transactions do not have a failure state, unlike ethereum. They are either mined, or they are not.

In general, a transaction should be considered final when:

It is included in a block, let's say with block number n
There are 5 blocks after this (current height >= n+5)

You don't really even need to loop every second. You can use the zmq feature of Bitcoin Core, or one of the numerous websocket APIs, or just poll for new blocks every 10 minutes. Once you see your tx in a block, you have one confirmation. Then, you simply need to check the next blocks to make sure they build upon the block with your tx.


Bitcoin Core reports progress only up to 0.999999 On older version - after full synchronization in debug log i see message
progress=1.0
In new - only progress=0.999999, it`s normal or i have some problems with synchronization?

This is a rounding issue, and not new.

You are most likely fully synchronized.


How net earners got their incomming channels? Lighting looks good for net spenders:

download node/wallet
lock your funds by openning channel to some hub
spend until has something in channel
close it

All costs like channel fees and locked funds goes to funder/spender.

But what in case of net earners (shops like bitrefill) another party should lock their funds and pay fees to deliver funds. I don't found answer in Lighting RFC, it seems like it should be whole OTC market with Banks/Hubs and unsafe protocols. Net earners should pay someone to lock their moneys by opening channel in their direction.

Inspired by:
https://medium.com/failsafe/why-lightning-and-raiden-networks-will-not-work-d1880e4bc294

Merchants get channels from customers who want to transact with them. This isn't something that needs to be specified; if someone wants to transact with you over LN, they can open a channel with you. This is how many transactions currently happen over LN - the customer opens a LN channel with the receiver. Furthermore, autopilot modes like the one used by LND will automatically open and close channels to various nodes so nodes that accept incoming channels will likely have connections from others where autopilot connected to it.


How to fetch all UTXO from a SPV wallet using bitcoin-filter.js (or other lib)? I am using bitcoinjs-lib to create private key (WIF), addresses and sign the transactions.

I would like to fetch all transactions (UTXOs) associated with a wallet. I found these libs:  bitcoin-filter  and bitcoin-net .

But I didn't understand the code below:

var filter = new Filter(peers)
filter.add(new Buffer('818895f3dc2c178629d3d2d8fa3ec4a3f8179821', 'hex'))
filter.add(walletObject)

What parameters does filter receive ? Are they the addresses ?
If so, how can I find all the used addresses of a HD wallet ?

You are asking a few different questions here, I'll try to help with each:
Wallet filters: See BIP37 which describes SPV wallets in general. A Bloom Filter is generated by the SPV wallet and sent to full nodes on the network. The filter is a compact data structure that represents every public key the SPV wallet is interested in. The full node will check each transaction in blocks and the mempool against this filter and if there is a match, that transaction is sent back to the SPV wallet. BIP37 specifies all the data the filter is tested against but in summary, it includes transaction hashes, public keys, public key hashes, and script hashes.
You can see here in the bcoin wallet (also JavaScript) where the wallet's public key hashes and watched outpoints are added to the bloom filter.
The bcoin socket API also has a method for adding address hashes to a bloom filter before it is sent to a full node.
HD Wallet Addresses: To understand HD wallets it's important to read BIP44 which describes the recovery process. During BIP44 wallet recovery, the program starts with a seed of entropy (derived from your 12- or 24-word phrase). It then generates account 0, receive and change addresses 0 through 20. Then, starting from a specified block height (just before the wallet was first generated in time) it iterates through the blockchain looking for TXs that match those addresses. Every time it finds a match, it generates another address (the 21st address, and so on...). In SPV mode, each new address would have to be added to the Bloom Filter and sent back to the full node. This actually can cause issues with the wallet sync, and some blocks may need to be re-scanned.
Notes:

I am using bitcoinjs-lib to create private key (WIF), addresses and sign the transactions. I would like to fetch all transactions (UTXOs) associated with a wallet.

In general using a single private key and reusing a single address is considered bad practice. BIP44 is an excellent design for wallets that generate multiple keys and addresses.
If you really only have one address to scan for UTXO you can use an address indexer like in bcoin or soon-to-be-merged to Bitcoin Core. You may also have some success just using rpc scantxoutset fo your single address.


Is there a database of all transactions? The blockchain is a database of all valid transactions, but is there a company/project that also logs all transactions which were broadcasted, but didn't made it to the chain? Blockchain.info for example, keeps a history of all double-spends, so I assume they keep such a log, but its not publicly available.

I could very easily calculate how much was lost by transaction maleability if something like that was publicly available.

I haven't seen anything that is tracking all transaction including failed ones, etc although  this project looks like it has promise:

http://codesuppository.blogspot.co.uk/2013/07/work-in-progress-on-bitcoin-blockchain.html

That said I'm not even certain what your suggesting would be possible.  Consider the following scenario.

If a person / organisation were monitoring and recording all transactions and I were to submit a transaction that then where then to be marked invalid and discarded, the nature of bitcoin is that you send your transaction to peer nodes who in turn will broadcast those out if they verify the transaction.  Now If someone can explain to me otherwise than fine but surely this transaction would only have a limited number of broadcasts before a different chain showing it to be be invalid was received and the erroneous block discarded.  Now unless the 'monitoring party' where within the peers that received the broadcast there would be no way for them to track it?


How is spamming of the block chain discouraged by the Bitcoin protocol? I have heard people imply that there are parties that want to spam the block chain for some reason, for example to get more support for higher blocks.

How is spamming the block chain discouraged by the Bitcoin protocol?

It's not really.

Of course there are transaction fees, but in the situation you describe, where blocks are not already full, the supply of available block space must be lower than the demand, and so you should be able to create many transactions at low fees, and miners will put them in their blocks because low fees are better than none at all.

Or, if you are a miner yourself, or have a deal with one, you can fill up your blocks with your own transactions, with whatever fees you like, since they are all going back to you.  (There is a slight risk here: if you make the fees significantly higher than the market rate, the next miner may choose to orphan your block and include those transactions in her own; then she gets to keep the fees you were planning to pay back to yourself.)

There is a slight disincentive for such a miner in that large blocks propagate slower and thus are slightly more likely to be orphaned.

Other miners might notice you are doing this, if you appear to be getting way more transactions in your blocks than anyone else, and so it might not help convince them to support larger blocks.  But it doesn't really prevent the "spam".


Critical Section in Satoshi client Can you please point me to a tutorial for an explanation of the specific "Critical section" implementation that is used in main.cpp. I would like to understand a bit better how does it work. I studied concurrency before so I have some background to be able to understand it.

Are you talking about the CCriticalSection class?  As you can see in sync.h, it's just a wrapper around Boost's recursive_mutex class.  (It looks like the CCriticalSection name comes from Windows.)

You can read all about Boost's threading API in its documentation.