Why are hashes base16 (hex) encoded rather than base58 encoded? BTC sha256 hashes are hex encoded (as sha256 spits out bytes not a string), why is this? Surely using base58 would be more efficient as would use less space (would output a shorter string)

Mark H's answer is correct, but I think there is a more fundamental point: addresses are directly intended for human consumption. Transaction hashes (as well as public keys, block hashes, raw scripts, raw transactions, ...) are not. They are sometimes visualized, but this is primarily for debugging purposes. And hexadecimal is a very natural choice for debugging, as it's easy to see which bytes correspond to which characters.


How do I calculate the txid of this raw transaction? I'm trying to parse a bitcoin block using Python.

I called getrawtransaction over JSON-RPC. I got this:

01000000010000000000000000000000000000000000000000000000000000000000000000ffffffff0704ffff001d014dffffffff0100f2052a01000000434104e70a02f5af48a1989bf630d92523c9d14c45c75f7d1b998e962bff6ff9995fc5bdb44f1793b37495d80324acba7c8f537caaf8432b8d47987313060cc82d8a93ac00000000

How do I calculate the txid of this raw transaction?

TXIDs have two forms; the form used internally for outpoints and merkle leaves (internal byte order), and the form used in RPC calls and block explorers (RPC byte order).  These are often called, respectivey, little-endian byte order and big-endian byte order (although those descriptions aren't quite accurate).

Here's python code from the Bitcoin.org Developer Documentation generating both forms for the TXID for the transaction in your question:

#!/usr/bin/env python

from sys import byteorder
from hashlib import sha256

## You can put in $data an 80-byte block header to get its header hash,
## or a raw transaction to get its txid
data = "01000000010000000000000000000000000000000000000000000000000000000000000000ffffffff0704ffff001d014dffffffff0100f2052a01000000434104e70a02f5af48a1989bf630d92523c9d14c45c75f7d1b998e962bff6ff9995fc5bdb44f1793b37495d80324acba7c8f537caaf8432b8d47987313060cc82d8a93ac00000000".decode("hex")
hash = sha256(sha256(data).digest()).digest()

print "Warning: this code only tested on a little-endian x86_64 arch"
print
print "System byte order:", byteorder
print "Internal-Byte-Order Hash: ", hash.encode('hex_codec')
print "RPC-Byte-Order Hash:      ", hash[::-1].encode('hex_codec')

And here's the output:

Warning: this code only tested on a little-endian x86_64 arch

System byte order: little
Internal-Byte-Order Hash:  660802c98f18fd34fd16d61c63cf447568370124ac5f3be626c2e1c3c9f0052d
RPC-Byte-Order Hash:       2d05f0c9c3e1c226e63b5fac240137687544cf631cd616fd34fd188fc9020866

The internal byte order used by the Bitcoin is in fact Little-Endian, while the RPC byte order is Big-Endian. For details explaining how this works refer this link.


Why do we need alternatives to Bitcoin? I don't understand why people need to create alternatives to Bitcoin such as litecoin. Why were these created? What's the point? You can't answer those questions using the same arguments for why Bitcoin was created.

In addition to Lohoris's answer, the reason that litecoin was created was to be as resistant to custom built hardware just for mining the coins, it uses scrypt as its cryptographic function. Scrypt was specifically designed to make it as expensive as possible to create hardware that is specifically good at doing that hash function. I personally like this because I do not like the idea of buying hardware that can only do SHA-256 to mine cryptocoins.

There is also ppcoin (pdf warning) which is aimed at not relying on constantly using electricity to keep the network secure. In this way it is very distinct from bitcoin.


Compiling litecoind on Ubuntu 15.04 I am trying to compile litecoind on Ubuntu 15.04 and I stuck here:

/litecoin/src$ make -f makefile.unix USE_UPNP=-
make: makefile.unix: No such file or directory
make: *** No rule to make target 'makefile.unix'.  Stop.

I found some a little bit different ways to compile litecoind, but every time I have problems with "makefile.unix". Can someone help me?

The build system has changed to autotools in later versions of Litecoin.

The process now looks like this:

./autogen.sh
./configure
make -j4
make install # optional

There's up to date instructions in detail in build-unix.md.


Bitcoin version conundrum Given that the Bitcoin node software not centrally controlled (auto updated) and many different versions and implementations exist around the world, how would a core change be possible if it is not forced to all nodes? Example one would want to make the block size larger or change the hashing algorithm to be resistant to newer technologies like quantum computing attacks, not everything can be made backward compatible. How can one ensure that the newer improved core makes it to the many nodes that exist.

how would a core change be possible if it is not forced to all nodes?

Simple, it can't be. You can't force a change, users must willingly adopt it.

The bitcoin network is very conservative in regards to changing network rules, especially the rules that keep the nodes in consensus. This is by design: users that engage with the network have a high degree of certainty that no single actor (or group of actors) can force a change on the network. You could say that the rules of bitcoin are enforced by the code, not by the whims of some future politician. As it is often said, the inherent difficulty to change bitcoin's rules is a feature, not a bug.

There are specific details depending on the rule change in question, but in general: making a change to the bitcoin network requires an overwhelming majority consent. To avoid messy fork situations, changes need to be planned carefully and accepted pretty much globally.


Can a single transaction have multiple outputs going to the same address? I understand that a transaction can have multiple outputs. But can two outputs  in a transaction refer to the same target address?

Here's how this would like in pseudo style:

[vout] => Array
        (
            [0] => Array
                (
                    [value] => 0.01
                    [scriptPubKey] => Array
                        (
                            [addresses] => Array
                                (
                                    [0] => bitcoinaddress_xyz
                                )

                        )

                )

            [1] => Array
                (
                    [value] => 0.02
                    [scriptPubKey] => Array
                        (
                            [addresses] => Array
                                (
                                    [0] => bitcoinaddress_xyz
                                )

                        )

                )

        )

Here the same address "bitcoinaddress_xyz" appears in both outputs.

I know that it would not make much sense to construct a transaction like this – I just would like to know if it is technically possible.

Thanks.

It is possible. Here is an example: http://blockchain.info/tx/4199f709bfab79f54938e80e11dc75c21a95e3183f8e8994306f5ff272fde42a

However, the bitcoin-qt GUI doesn't allow creating such transactions.

Such transactions might be useful in the context of colored coins, or where encoding information is involved.


Wouldn't it be profitable to make an ASIC capable of calculating private keys? Just looking at vanitygen-plus (currently supporting almost one-hundred crypto-currencies) do its stuff, and it is capable of over 548,990 keys/sec on my CPU alone.

Well developed ASIC is several orders of magnitude faster than my CPU, wouldn't it be profitable for a company with the resources to do so to develop such a key miner, even if it is a one-off?

The reward (actually, theft) is access to the private keys of any address. Surely the development costs are not so prohibitive for ASIC as they are for general-purpose CPU's?

Future developments in this area seem like the most realistic threat to crypto-currency security so far.

You could in theory build an ASIC to brute force private keys of any cryptographic scheme, like EC or RSA.

But ASICs only give linear improvements. That means that they 'accelerate' some search only by a constant factor, like 2, 3 or 100. Even if your ASIC does something a million times faster than a normal CPU, it's still a linear improvement.

If guessing a private key takes the current age of the universe in a CPU, an ASIC a million times faster would still take 13 thousand years.

Cryptography relies on problems being exponential on the key size. Exponential growth beats linear growth. If some day a powerful ASIC or CPU could guess a ECDSA private key, users of ECDSA would only have to increase their key sizes.

To yield a better than linear improvement, you would need a better algorithm. Implementing current day algorithms in faster hardware only gives linear improvements. It's widely believed, even if we still don't have formal proof, that it's not possible to get better algorithms for prime factorization and elliptic-curve factorization. (At least in non-quantum computers.) In other words, they are considered intractable problems.


Bitcoin Core 'unable to open file' causes reindexing to fail I recently struggled to send bitcoin from my Armory wallet (0.96.5) to another account.  Sending bitcoin did not work because the wallet could not synchronize with the blockchain.  Since Armory relies on the connection managed by the bitcoin core client, bitcoin-qt or bitcoind, it became evident that the failure was caused by the bitcoin core clients (0.17.1). The command ./bitcoin-qt --reindex was issued after Armory was shutdown, but failed repeatedly with entries in debug.log having excerpts such as the following immediately before shutdown:

2019-01-15T23:17:07Z Pre-allocating up to position 0x1000000 in rev01636.dat
2019-01-15T23:17:07Z UpdateTip: new best=0000000000000000001e1db68a762512729668e9dd4c6c06cae346a65618189e height=557921 version=0x20c00000 log2_work=90.224133 tx=371925681 date='2019-01-10T12:31:34Z' progress=0.996972 cache=700.2MiB(5254262txo)
2019-01-15T23:17:07Z UpdateTip: new best=00000000000000000014b6677c848def28a7edef1225b0aa865f96199cb1abd8 height=557922 version=0x20000000 log2_work=90.224157 tx=371928196 date='2019-01-10T12:42:46Z' progress=0.996976 cache=700.3MiB(5254902txo)
2019-01-15T23:17:07Z Imported mempool transactions from disk: 1 succeeded, 0 failed, 0 expired, 0 already there
2019-01-15T23:17:09Z Leaving block file 1636: CBlockFileInfo(blocks=144, size=133976289, heights=557764...558003, time=2019-01-09...2019-01-11)
2019-01-15T23:17:09Z Unable to open file /home/**/.bitcoin/blocks/blk01637.dat
2019-01-15T23:17:09Z Unable to open file /home/**/.bitcoin/blocks/blk01637.dat
2019-01-15T23:17:09Z ERROR: WriteBlockToDisk: OpenBlockFile failed
2019-01-15T23:17:09Z *** Failed to write block
2019-01-15T23:37:10Z socket sending timeout: 1201s
2019-01-15T23:37:10Z socket sending timeout: 1201s
2019-01-15T23:37:10Z socket sending timeout: 1201s
2019-01-15T23:37:10Z socket sending timeout: 1201s
2019-01-15T23:37:10Z socket sending timeout: 1201s
2019-01-15T23:37:10Z socket sending timeout: 1201s
2019-01-15T23:37:10Z socket sending timeout: 1201s
2019-01-15T23:37:10Z socket sending timeout: 1201s
2019-01-15T23:37:10Z socket sending timeout: 1201s
2019-01-15T23:37:10Z socket sending timeout: 1201s
2019-01-15T23:39:47Z ERROR: ProcessNewBlock: AcceptBlock FAILED (AcceptBlock: Failed to find position to write new block to disk (code 0))
2019-01-15T23:39:47Z tor: Thread interrupt
2019-01-15T23:39:47Z Shutdown: In progress...
2019-01-15T23:39:47Z addcon thread exit
2019-01-15T23:39:47Z opencon thread exit
2019-01-15T23:39:47Z torcontrol thread exit
2019-01-15T23:39:47Z net thread exit
2019-01-15T23:39:47Z msghand thread exit
2019-01-15T23:39:47Z scheduler thread interrupt
2019-01-15T23:39:47Z Dumped mempool: 3e-06s to copy, 0.000992s to dump
2019-01-15T23:39:47Z Unable to open file /home/**/.bitcoin/blocks/blk01637.dat
2019-01-15T23:39:55Z Unable to open file /home/**/.bitcoin/blocks/blk01637.dat
2019-01-15T23:39:56Z [default wallet] Releasing wallet
2019-01-15T23:39:56Z Shutdown: done

What can be done when a failure to open a file causes reindexing to fail?

An exhaustive survey of Google searches relating to the failure to read blknnnn.dat files implicated data corruption on the hard drive or in files such as wallet.dat and/or blknnnn.dat.  In my case, however, I was able to rule out these factors.  I ruled out hard drive failure by using a clone of my virtual machine that ran on a substitute hard drive.  I also ruled out corruption in wallet.dat by reindexing a substituted wallet.dat that had recently worked fine.  In this case the file that could not be opened was blk1637.dat. After about four unsuccessful reindexing attempts, I looked at the file permissions for the defective block and noted that file permissions were only allowed for root user. Prior blocks were permissioned for a non-root user.  The obvious solution to this lack of uniformity in file permissions was to conform the file permission settings of the errant file, but alas making this change and then reindexing the chain did not work, for the same failure to open the block file happened again.  I noted a comment someone made in a Google search that init.cpp in the client software used the umask command to change file permissions for the block files.  Using the sudo command saved me in this situation.  After all, I have everything contained within a virtual machine environment and this security risk was nominal.  In the appropriate folder, I ran sudo ./bitcoin-qt, waited for it to catch up with the syncing, then I started up Armory.  Ordinarily, using sudo in order to force a resort to root privileges is not a good idea.  But in this case it enabled me to transfer my bitcoin to a hardware wallet, freeing me up to run a new node on a virtual machine where the block data files have a consistent 6 0 0 file permissions setting unaltered by an errant system shutdown.


Zero to Bitcoin in sixty seconds - how to set up a Bitcoin client in case you need it fast? Lets say I have a bitcoind instance running on a server with proper wallet.dat backups created regularly. The server crashes, blockchain data becomes corrupted or what have you, and I need to set up a server client fast to take over traffic. Moreover, I need the service to be interchangeable with the official client - that is it needs to provide the same RPC commands, conform to the wallet.dat specification so it can be interchanged between the clients and so forth. What is the fastest way one can do that in case of emergencies?

Well, I would run the bitcoind in a VM, and regularly snapshot it.  If it fails, rollback the snapshot and reboot. Back online in < 10 min, assuming frequent backups and you start working right away.  Not sure if this is the use case you're asking about, but that's what I'd do for max uptime.


What will happen when the time field wraps around I was rather surprised when I read about the bitcoin's block header time field is only 32 bit. Even if it is unsigned (it is) it only get us 68 more years than the Y2K38 problem. What will happen then, will we need to change the protocol? Can we? What would it imply then?

Why at first Satoshi did not put a 64 bit timestamp at first was that intended?

2^32 seconds is 136 years. So, this will become a problem in the year 2106. However, even at that point, there are things that we can do to fix the problem.

While it is technically correct that the block number is not encoded in each block header, the block number can easily be inferred. Each block header has the hash of the previous block header. If you follow this back far enough, you'll get to the genesis block. You don't even need to download the entire transaction history, just the block headers.

Okay, so we've got the block number, and the date field. Now, we make a rule that on block 4786000 and all later blocks (which will land at about 2100) the date field will be interpreted to be 100 years later. Block 4786000 will state a date of 2000, but because we made this rule well in advance (we did make it well in advance, right?) all of the clients will know that the date actually means 2100.

Thus, we have bought ourselves another 100 years. On block 9572000, we can do exactly the same thing.


Bitcoind, change to txindex=1 I have a fullnode without txindex.
Some transactions of mine related to my wallet are not being found by fullnode, only in blockchain.info.
My HD is around 130GB, and it's fully synced to the last block.
I would like to know, if using txindex = 1, will solve my problem with transactions that are not found.
Using txindex = 1 and -reindex, how long does it take to finish?
Fullnode will have to download again the 130GB? Will increase storage?

For exemple:
https://blockchain.info/pt/tx/1585045cf36e18829292eef72281c8a0a748488d960816437dbd0797d896adf9
This transaction send for 1BS3caAYJH5LTKprkbvhkSLVUhpa9QsvMp my wallet.

When i run bitcoin-cli getreceivedbyaddress 1BS3caAYJH5LTKprkbvhkSLVUhpa9QsvMp 0, result is 0.00000000
When i run bitcoin-cli gettransaction 1585045cf36e18829292eef72281c8a0a748488d960816437dbd0797d896adf9, resulti is error code: -5 error message: Invalid or non-wallet transaction id
How i solve it ?

Thank you.

I would like to know, if using txindex = 1, will solve my problem with transactions that are not found.

Yes, getrawtransaction only works for transactions with unspent outputs, unless txindex is enabled (in which case it will find every transaction). gettransaction only works for your wallet transactions.

  Using txindex = 1 and -reindex, how long does it take to finish?

It depends on your hardware, but it can be between hours and days on low-end hardware. Using a larger -dbcache will massively speed it up.

  Fullnode will have to download again the 130GB?

No. It just reprocesses blocks you already have on disk.

  Will increase storage?

Yes, a few gigabytes.

  OBS: My real problem is that there are some transactions that have been made to my wallet and my fullnode can not find it.

Your wallet has nothing to do with getrawtransaction. You can always use gettransaction for your own transactions, though.


Can you send a transaction from non-native segwit to native(bech32) segwit address? Let's say I have a wallet with a non-native segwit address (prefixed with 3- like multisig).

Can I send my utxos from this address to a native segwit (bech32) address?

The wallet needs to be able to decode a bech32 address and construct the appropriate native witness output when sending funds.

Given that your (hypothetical) wallet can spend P2SH(witness) outputs already, it is likely it can or will soon support native bech32 addresses, but there is no guarantee that the latter must be supported given the first.


Negative time estimation from estimatefees + mempool data I am currently writing a time estimation tool for Bitcoin, and it gives me wrong estimations, so I am not sure if it's accurate and if I am not doing some error in logic.

For example, I have this transaction

https://blockchain.info/tx/388f99d51cfce4f18f0c76257fff7666ffd272394ee0316df529383c6d22ed66

It has 10 sat/B (both virtual size and real size). It has 0 ancestors in mempool, all its inputs are confirmed. (all from getmempoolentry call)
Estimatesmartfee returns 11.089 sat/B for 192 blocks, that is 1 day and 8 hours.
The transaction was seen about 4 days ago (again, according to time in its mempool entry) and is still unconfirmed.

That means it should have been confirmed 3 days ago. So its estimated confirmation time is -3 days, which is impossible.

What am I doing wrong?

Fee estimation is mostly progress free, in the same way as waiting for Bitcoin blocks, that is the estimate for how long it will take to get confirmed from now is unrelated to how long you have been waiting.  I say mostly, because there are clearly daily and weekly cycles in transaction activity that result in the fact that overnight or over the weekend there are greater chances of getting confirmed.

Aside from that, fee estimation in Core is purely backwards looking, and can only tell you what happened in the recent past to transactions of a certain fee rate.  It can not predict the future.  If 2/3 of the hash rate disappears as is the case with the transaction you presented, then it is to be expected that it could take many more blocks than the estimate had previously predicted.

Also, Core fee estimation tries to give a fairly conservative estimate, so in theory you should most of the time be confirmed in less than the target number of blocks. So you could ask for current estimates until you find one which is lower than the fee rate in question and this should give you some guess as to how much longer you might need to wait, but again it can't predict the future.

About only the definitive thing you might be able to get, is if you know you placed the transaction X blocks ago and if you ask for a fee estimate for X/2 blocks now and the answer is less than the fee rate you put on the transaction then it is reasonable to question why your transaction has not been confirmed yet.


What are the benefits of Invertible Bloom Lookup Tables for Bitcoin? Gavin Andresen announced a few days ago, that he started working on Invertible Bloom Lookup Tables.

His repository's README refers to two papers, What's the difference?: efficient set reconciliation without prior context by Eppstein, Goorich, Uyeda, and Varghese, and Invertible Bloom Lookup Tables by Goodrich and Mitzenmacher, whose abstracts seem to be hinting at a special datastructure that could significantly improve lookup times, and might be expedient for consolidating information in a peer-to-peer network, e.g. block propagation.

Has someone already wrapped his head around the whole thing and would kindly explain:

What do Invertible Bloom Lookup Tables do?
How would they be used in the Bitcoin project?
What improvements compared to the status quo can we expect?

Invertible Bloom lookup tables can be used in many different ways, but the first paper you linked to explicitly proposes to use them to find differences between two data sets stored on different hosts without transferring the entire data set from one of the hosts to the other.

In context of Bitcoin, suppose that two nodes have just established a connection. Now they need to synchronize their transaction pools to make sure that each node knows about every transaction the other node knows about. A trivial solution is to send the list of transaction IDs for all transactions in the pool from one node to the other, but this list can be large. At the same time, each node will probably have most of the transactions that the other node has, so the number of transactions that have to be transferred is small.

The problem is to find which transactions should be transferred, and this is the problem that invertible Bloom lookup tables can solve. Rather than sending the list of all transactions IDs, a node only sends an invertible Bloom table, which is smaller, but it is only usable if the number of differences (transactions which are present on one of the nodes but not on the other) is small. The table can be made of different sizes, and the larger the table, the greater is the number of differences that it can resolve.

For Bitcoin, this means that the amount of unnecessary communication will be reduced, and that nodes will be able to synchronize their transaction pools more quickly. Bitcoin nodes need to synchronize the block chain as well, but because of its linear, append-only structure, there are simpler and more efficient algorithms for this, and such algorithms are used in Bitcoin from the beginning.


Does pruning node keep old wallet's transactions? Does pruning node keep old wallet's transactions? For example I need to use the gettransaction API to retrieve a wallet's transaction in a deleted block, should I use the pruning mode? I cannot find any document about it,
Thanks.

Yes, it does.

Pruning only affects the blockchain/validation logic. The wallet always maintains a copy of all transactions relevant to it (including e.g. unconfirmed transactions that the validation logic does not know about).


Why do transactions take so long to confirm in testnet? I have created a transaction with high miner's fee and it is now in mempool (in testnet):

https://live.blockcypher.com/btc-testnet/tx/abcc4ce473f490896e87102fe3ffb857ff735a0fc97525dadf9c5b0ed5410f9f/

However, it doesn't seem to be confirming. It's already missed two opportunities to confirm. Transactions with much lower miner's fees are going through instead.

Is Bitcoin this random or am I missing something?

Decoding your transaction shows an output:

"vout": [
    {
      "value": 0.00000000,
      "n": 0,
      "scriptPubKey": {
        "asm": "OP_RETURN 68656c6c6f5f776f726c64",
        "hex": "6a0b68656c6c6f5f776f726c64",
        "type": "nulldata"
      }
    }, ...

This might be the reason the miners won't touch it as it may be flagged as non-standard on older nodes. Notice the warning on blockcypher:

Maybe someone can comment on exactly why, as these null_data transactions were made standard on Bitcoin-core 0.9.0, see https://bitcoin.org/en/developer-guide#null-data


How to configure the official bitcoin client to conveniently use offline wallets? This is a follow up question to "How can I keep my wallet secure?"

Is there a portable (at least Win32&64/Linux) to configure the official bitcoin client to use only offline wallets to be inserted in the usb slot for each transaction?

You can use symbolic links to your bitcoin data folder or wallet.dat to the usb device location.  You will still have to run bitcoin -rescan when you start the bitcoin client, and you will have to remove the link if you want to run bitcoin without the usb devices.

Note, however that you do not need to have an offline wallet online at anytime ever unless you are spending the money out of it.  If you are only sending money to it then it can remain offline.


What are colored coins? What exactly are colored coins? I've tried understanding, but I don't quite get it yet . . . Do any examples exist?

Colored coins are a method to track the origin of bitcoins, so that a certain set of coins can be set aside and conserved, allowing a party to acknowledge them in various ways. Such coins can be used to represent arbitrary digital tokens, such as stocks, bonds, smart property and so on.

The colored coins protocol is decentralized just like Bitcoin, but the current effort to develop an implementation is done under the BitcoinX project (tentative name), which also aims to provide some related services.

As this is still under development, you will not find any existing examples.

You can read more about it in Overview of colored coins (work in progress).


Bitcoin hard disk replace to a solid disk So, i might change my hard drive disk to a solid disk on my mac. Before I do that, what should I do to have my bitcoins safely saved from the hard disk to the solid disk once I install the solid disk in my mac?

You have to copy your wallet files, which contains the private keys. In case of Bitcoin Core, you need to copy a file wallet.dat. In case of electrum, you just need a seed, unless you also care about transaction history labels and saved "contact" addresses, than you should backup your default_wallet file.


Has the Timestamp Server from the original whitepaper ever been developed? In the original whitepaper Satoshi proposes a creation of a timestamp server to authenticate that Bitcoin blocks were not created later than at a given time. Has this feature been ever implemented or planned to be implemented? Why would the timestamp server be useful to the Bitcoin client/protocol/community?

The system called timestamp server in the original whitepaper is currently more often referred to as a "miner".

They timestamp blocks, which combine several transactions, imposing an authorative order on these transactions. This is necessary as the system cannot otherwise decide which of two conflicting but otherwise valid transactions should be accepted.

It's certainly not a general-purpose timestamping system, but that can be built on top of it. See for example ChronoBit or OpenTimestamps.


How much escrow/ Multi-signature transactions are done? I have read here about the possibility of Multi-signature transactions. Is there a statistic available how much of these are done?

Blockchain doesn't currently offer that and the only way to construct a transaction like that is to build it manually.  So as a result both the sender and each recipient must have l33t skills.

This capability will come, it is just that since that was introduced, scalability had to take priority over new features for most Bitcoin development.

They happen but rarely.  Here's an example of Bitcoin address used recently (March 5, 2013) in an escrow transaction:

,- http://blockchain.info/address/13MH4zmU4UT4Ct6BhoRFGjigC8gN9a9FNn


In the original source code there were little restrictions on what could be in the coinbase data of the coinbase transaction. I think the only restriction was that the coinbase data should be between 2 to 100 bytes. However, this lack of restriction on what can be within the coinbase data brought up a vulnerability. It was perfectly legal for miners to create a coinbase transaction identical to previous one and mine it in a block at a height higher than where the "original" coinbase transaction. When this happened it led to the invalidation of the previous transaction.

This was fixed later when couple of blocks were mined with identical coinbase data. As per BIP-34, version-2 blocks must contain the block height index as a script push operation in the beginning of the coinbase field. For example, in your coinbase transaction this data is represented by the bytes 0x03400d03. The first byte 0x03 indicates the script should push the next 3 bytes. The next three bytes 0x400d03 represent the block height 200,000 in little endian.

The rest of the coinbase data is free for the miners to use as per their will. This is mostly used as a space for extra-nonce. In the early days of Bitcoin, a miner could find a block by iterating through the nonce until the resulting hash was below the target. As difficulty increased, miners often cycled through all 4 billion values of the nonce (232) without finding a valid block header hash. However, this was easily resolved by updating the block timestamp to account for the elapsed time. Because the timestamp is part of the header, the change would allow miners to iterate through the values of the nonce again with different results.

Once mining hardware became really efficient, however, this approach became increasingly difficult because the nonce values were exhausted in less than a second. Hence, the mining software needed more space for nonce values in order to find valid blocks. The timestamp could be stretched a bit, but moving it too far into the future would cause the block to become invalid. The solution was to use the coinbase transaction as a source of extra nonce values. Because the coinbase script can store between 2 and 100 bytes of data (~first 4 bytes reserved for block identification), miners started using that 96 byte space as extra nonce, allowing them to explore a much larger range of block header values to find valid blocks.

The mining pools also use the coinbase data as an identification of who mined the block. For example if you look at block 594,041, you will see the coinbase data contain the bytes 0x4d696e656420627920416e74506f6f6c. It represents "'Mined by AntPool" in ASCII. (Unfortunately the block you mentioned did not have any identification hence I had to use a different block that did have it). However, it is important to note that this data cannot be relied upon, as any miner can mine a block tomorrow with a data claiming it to be mined by a different pool, may be to hide its identity or frame others.


How are bitcoin transactions and the blockchain transferred over the internet? So I have several specific question connected to this topic and I'm aware of the conceptual answers and I'm only interested in the technical details (with programming terms and concepts as I'm a developer).

My questions are:

If I'm sending a transaction from my wallet application (e.g. electrum), what does it do under the hood? Does it use the HTTP protocol or something else? Or not even use TCP but UDP instead?
How does the miners "listen" to transactions? Do they listening on some port? Is there any generic protocols for "listening" like this?
When a miner mines a block how does he/she broadcast it to the network? Is it the same as sending a transaction from my electrum wallet?

The Bitcoin P2P network

The Bitcoin P2P network is a randomly-wired gossip network. This means that all nodes make arbitrary connections to other peers (using various ways to discover new addresses) using a custom TCP protocol, usually using port 8333. Typical nodes create 8 outgoing connections, and if publicly reachable, accept up to a few 100 incoming connections.

It being a gossip network means that nodes send messages to each other of the form "do you have block with hash X" or "do you have transaction with hash X", after which the peer can respond by asking for the full transaction or block. Simplified, this means that eventually every node learns about every block. The protocol is bidirectional - it doesn't matter who connected to whom; both peers will announce if they have something they don't know the other side already has as well.

Not all network nodes are reachable. Some only make outgoing connections. Sometimes setups are used where someone runs a publicly reachable node, and then runs a second private node that only connects out to their own public node, thereby giving some extra privacy and reduced attack surface.

Not all network nodes validate the blocks and transactions they see and pass on. Nodes that do are called full nodes.

Miners (or at least mining pools) run nodes like this, picking up transactions their nodes learn about, assembling them into candidate blocks, hashing to find a nonce, and if they succeed, broadcast the resulting block out to their peers using the same P2P network. Miners in this way are not special - people don't directly submit things to miners.

Since recently, an extension to the protocol is used to relay blocks much more efficiently, called Compact Blocks. It makes use of the fact that most nodes will already have most of the transactions that are added to a new block.

Specialized relay networks

Additional networks exist of groups of nodes that use specialized protocols to very quickly interconnect nodes. One such protocol is FIBRE, which is in fact UDP based and makes heavy use of error correcting codes to avoid roundtrips.

Electrum

Ironically, Electrum in your question is a bad example. Electrum is not a P2P client (not even a light one). Instead, it uses a client-server architecture, and connects to specialized servers (which are themselves Bitcoin P2P nodes however).


Hack/Use a Butterfly bitcoin miner as gaming pc How can I hack & build upon (i.e. add DVD drive, USB reader, etc.) a BitCoin miner like (Butterfly Jalapeño).

Everyone tries to use their gaming computer as miner but I want to venture & try to get insane performance by using parts of a miner & build a PC from it.

Or it is impossible?

This is not possible.

ASIC mining devices like those sold by Butterfly contain hardware that is capable of computing SHA256 hashes at high speed, but nothing else.  They are not general purpose CPUs and cannot be reprogrammed to run other code.

They might contain a simple microprocessor or microcontroller to control the ASICs and interface with the host computer, and in principle you could run other code on that, but it would have to be your own custom code as these chips are presumably not x86, and the performance is probably too low to play anything more complex than Pong.  Moreover, these devices have no graphics hardware and no obvious way to connect it.  It would be like trying to play Call of Duty on your microwave oven.


Which block header fields are miners able to change in an effort to avoid having to recalculate the Merkle Root? In addition to the nonce and the Merkle Root, there are 4 other fields in the header - version, previous header hash, time and nbits.

Assuming a miner does not want to change the merkle root because of the additional computation involved, and obviously cannot change the previous header hash, to what extent are they able to change the version, time and/or nbits field to get additional nonce space?

nBits cannot be changed at all. Its value must equal the (compact encoding of) the target value for that block, which is determined entirely by the predecessors of the block. In other words, if the prevblock hash is given, so is nBits.

nVersion: can be modified arbitrarily, though its value is subject to interpretation under BIP34 (requiring it to be at least 2), BIP66 (requiring it to be at least 3), BIP65 (requiring it to be at least 4), and BIP9 (assigning potential future meanings to certain bits if the value is within a specific range).

nTime: can be rolled within a window starting at the median of the past 11 blocks + 1 up to 7200 seconds in the future at the time the block is mined.


new Bitcoin Core (0.13.2) as Cold Storage Wallet? What are the instructions (in GUI or CLI) to setup and use the new Bitcoin Core (0.13.2) as Cold Storage Wallet? OS is Ubuntu 16.04.

Bitcoin Core does not officially support cold storage. That doesn't mean it can't be used for it, but if you don't know what you're doing you are likely to shoot yourself in the foot.


What are "routine escrow mechanisms"? The original Nakamoto paper states

  routine escrow mechanisms could easily be implemented to protect buyers

What is meant by "routine escrow mechanisms"?

There are certain sorts of escrow that are very difficult to implement without a trusted party. For example, an escrow system that only sends along the payment when the seller has provided proof of shipping would be impossible without a trusted third party.

However, if we change the escrow system a little bit, then we can implement it without a third party. For example, suppose we say that the buyer and the seller merely need to agree about where the bitcoins are sent. That is something that a decentralized network can check for.


How to get confirmation of transaction using Bitcoin j implementaion I am using BitcoinJ lib for creating wallet and transaction.But now I want to know the confirmation count.So, is there any method which can give confirmation count

Use Wallet.getTransactions(boolean). This returns a Set<Transaction>:

boolean includeDeadTransactions = true;
Set<Transaction> transactions = wallet.getTransactions(includeDeadTransactions);

for (Transaction t : transactions) {
    //Do something
}

You will get all transactions you can looping over so you get each transaction details.


how many miners should validate a block before its considered valid? How many miners should validate a block before it gets added to the blockchain?
Or does every miner add the blocks it receives to its local blockchain?

Miners don't really actively validate blocks during the mining process. Miners find valid blocks and broadcast them to the network. It is up to the full-validating nodes on the network as to whether a block is accepted or rejected.


Does a blockchain need a distributed hash table? In this question on StackOverflow, the OP is asking if there is a difference between distributed hash tables and blockchain. The answer to that question seems to indicate that there is no point of overlap between the two as they are for completely different purposes. However, I would like to clarify this point.

To the best of my knowledge, blockchain is a peer-to-peer implementation, which means that the entire blockchain is stored in every peer. However, if there is a need to broadcast consensus of a particular transaction, there is still a need to look up the other peers in some kind of table of addresses. Am I correct? (My underlying question is: In a blockchain, would all nodes or peers get the information about successful transactions simultaneously? It doesn't seem likely to me, and I'm trying to understand the protocol for doing this broadcast in a simultaneous, or near simultaneous manner).

No. Peer discovery does not use a distributed hash table and cannot use a DHT. Once you are connected to the network, there is no need for you to find more peers to connect to (and most nodes have a limit of 8 anyways as that is Core's outbound connection limit).

The broadcasting of transactions and blocks themselves does not rely on the broadcaster to find all peers and send the data to each peer. Rather it simply sends the transaction or block to the peers it is connected to, and then those peers check the block and transaction, and if it is valid, they will send the block or transaction to their peers, and so on and so forth.


Did the Bitcoin fork resulting in the Bitcoin Cash created money from nothing? Since you can double-spend your old Bitcoin coins and the Bitcon Cash coins, independently---one on each side of the fork---it seems somehow that money was created for free. Isn't this a problem? What prevents people from creating more and more forks to the point of undermining the crypto-currencies value?

To answer the questions in the body of your posting:

Spending BTC "on one side of the fork" and BCH on the other cannot be called "double spending".
"Money created for free" is a misnomer. Yes, you have twice as many coins after a fork, but the combined value of both coins "right and left of the blockchain" is initially the same as just before the split, namely the coin with the former name has 100% and the one with the new ticker 0%. But from that point on price discovery on the market (exchanges) will determine the value of each strand individually.

"What prevents people from creating forks . . .?"
Among others the risk of loosing trust and competitiveness in regard to other crypto currencies.
If nevertheless a fork is triggered by some actors in a respective community, there must be important reasons why it couldn't be avoided. A fork is a declaration of war, after all efforts to compromise and reaching consensus failed.
But as we can see in case of Bitcoin, a fork does not necessarily undermine the value of that currency. The combined value of Bitcoin and Bitcoin Cash is now, 3 weeks after the fork, considerably higher than before.


How to avoid conflicts in a peer-to-peer topology (Bitcoin)? For example in Bitcoin, if I want a miner to verify whether other miners store a transaction, before deleting it from his mempool.

How can I handle the case more than one miner doing the verification at the same time?

Is it possible to use backoff algorithm?

The mempool is just a temporary store of unconfirmed transactions. Each node's mempool is not synchronized, and in fact a node could choose to not keep a mempool at all. So no node needs to verify whether anyone else has a transaction, before deleting it from their memory. Miners will choose transactions from their mempool, to build new block templates that they will mine on.

What matters is the blockchain. Each new valid block will (likely) include some transactions, so when a node hears about that block, it will update its mempool accordingly (removing transactions that have been confirmed in the block).

Thus, miners can all be attempting to mine a block that includes the same (or different) transactions, and it is simply the first miner to find a valid block that will 'win', with their block being appended to the blockchain. All other miners will see this new block, and start to mine on top of it, adjusting their mempool (and thus the transactions they include in their new attempts to find a block) accordingly.


So what exactly is my full node doing? I am aware of (I'm avoiding the term understand) the functions of transmitting/validating transactions/blocks but mainly trying to understand the resource utilization of my machine running the full node.

I have bitcoind running on Ubuntu16.04 Server so I have limited visibility into its activity beyond htop, iostat, etc. For the first time ever, I wish I was running Ubuntu Desktop and may do so unless you can recommend some tools for visibility into bitcoind activity.

But back to the subject, I currently only have 1 virtual CPU out of 8 available hyperthreaded cores on an i7 assigned along with 4G of ram, a single consumer grade disk, and a 60mb internet connection. I've confirmed inbound connections with bitcoind-cli getpeerconnectioncount to be 15-20 at any given time.

My disk thrashes like it could burn up at any time. My network connection constantly spikes up to 100%'ish of my inbound bandwidth such that I had to shut down bitcoind to watch a stream of a football game yesterday in decent res.

But my cpu and memory do almost nothing at all. Resource utilization is effectively nil.

So really an academic question as to what this node is doing to result in that footprint.

It is probably downloading the whole blockchain locally. It will take hours to complete.

the command  bitcoin-cli getinfo will display the information you're looking for.

If you want to see all available commands: https://en.bitcoin.it/wiki/Original_Bitcoin_client/API_calls_list


when is the mining of the last bitcoin? I have read online that the last bitcoin will be mined in the year 2140. However, I am not completely sure how this is calculated. As I have seen some sources say there is 32 Reward Eras and given that each reward era is roughly 4 years wouldn't the end date be:
2009 + 4 * 32 = 2137.
Thanks in advance for any explanations.

There will be 32 halvings, but the block subsidy will be reduced 33 times with the last reduction not halving it, but reducing it from 1 sat per block to 0 sats per block (see table with calculation). So there will actually be 33 eras with a block subsidy, which would mean that per your calculation the last new bitcoins would be created in 2141.
However, the first halving occurred in 2012, so that's how you get to 2140 if you assume 4 years per era. Actually, the average time per block since the Genesis block has been about 9.5 minutes (4,431 days for 671,456 blocks). In 2020, the average block interval was about 9.9 minutes. Assuming that an interval of 9.9 minutes would be maintained, the last block with a block subsidy would be found in December 2138.


How to store private data in blockchain and smart contract Following my previous question on this topic Is there a way to store and share a private key through blockchain?, I might have a solution but I would like to improve the process.

Image that A is the owner of some private data, and want to give the ownership of this data to B, another member of the blockchain. The data is a private key so it can not be public, but it can be encoded using A or B public key.

So:

Secret data is encoded with A public key, and stored in A informations data /
B want to by A secret data
A decode the secret data (using his private wallet key) and encode the secret data with B public key
Ownership of the data is given to B
Now only B (and A also but that's a different story) can decode secret data.

Is there a way to simplify this process using smart contract? I assume a smart contract have access to A and B public key, but can a data be encoded/decoded with private key of the one executing the smart contract?

If I've understood correctly, I don't think you need block-chain technology to achieve what you want. This is exactly what asymmetric public key encryption is designed for.

A simply encrypts some information using A's private key and B's public key... thereby ensuring:

Only B can decrypt the information
B can verify the information was
encrypted by A (using A's public key)

There's no way for A to prove to B that their copy of the information has been destroyed or forgotten by A, but at least secure transmission is ensured.


In a confidential transaction, what guarantees that the sender packed the correct ephemeral public key for the pederson blinding factor? I'm reading the Confidential Transactions paper, and it says that the blinding factor in the pedersen commitment is derived using a shared secret, but I can't understand what's preventing the sender from cheating a bit.

The sender generates a ephemeral private key, and derives the secret as (ephemeral private key * receivers public key), and is supposed to pack the ephemeral public key in the transaction, so that the receiver can derive the shared secret (receivers private key * ephemeral public key) and get the blinding factor and derive transaction amount.

What guarantees that the sender packed the right ephemeral public key? What's stopping the sender from packing a different ephemeral public key, preventing the receiver from figuring out the blinding factor and the transaction amount.

The sender might be up to mischief, or claim that he sent a transaction (and show the transaction ID as proof), and claim that the receiver (maybe an exchange) is not depositing the right amount.

What guarantees that the sender packed the right ephemeral public key? What's stopping the sender from packing a different ephemeral public key, preventing the receiver from figuring out the blinding factor and the transaction amount.

The fact that the receiver wouldn't treat the result as a valid payment.

This is no different from them sending the right amount to the wrong address.


Does Bisq somehow use the Bitcoin blockchain for recording trades? I just noticed that, after accepting a trade to buy BTC for fiat, Bisq says:

Deposit transaction has been published.
You need to wait for at least one blockchain confirmation before starting the payment.

I don't understand what any of that means. Before starting the trade, I pre-topped up my Bisq wallet with BTC for the security deposit, and it has (and had) six confirmations before I clicked "accept". So this text can't very well have anything to do with that.
I want to make clear that I have no practical problem using Bisq, and have used it many times in the past. I'm just wondering what they mean by that text, and whether Bisq actually uses the Bitcoin blockchain somehow. Maybe it's using automated "smart contracts"?

When fulfilling a trade the BTC seller deposit the coins in a multisig script with 2 keys, one controlled by them and one by the buyer.
When the BTC seller acknowledge the reception of the fiat payment, they release the coin to a script only controlled by the buyer.
This mechanism, as well as a number of others (see the documentation for details), aims to disincentivize fraud.


Why would increasing the Bitcoin block size lead to a more centralised system? Why does Bitcoin Core contributor Peter Todd think that increasing the Bitcoin block size would lead to a more centralised system?

“The system doesn’t scale, and you just have to accept that and do something smarter. Every time you do a [block size limit] increase, you’re making the system more centralized.”

How does Bitcoin block size affect centralisation?

The problem arises from the uneven propagation of information in the network. While the propagation of small blocks is fast, i.e., every miner gets to know a new block approximately at the same time, this is no longer true for large blocks.

Large blocks take a lot of time to be forwarded to every node in the network, since this includes miners, these miners are unaware that there is a new block and they'll continue mining on top of the old parent, i.e., they will work on a competitor of the newly found block. This means that they are actively wasting their computational resources on an obsolete solution if the propagating block gets accepted into the blockchain.

Big miners are likely to have a better connectivity with the Bitcoin network, resulting in them losing less of their computational time due. This in turn puts small miners at a disadvantage since they are now less competitive.

The extreme case would be one strong miner, with close to a majority of the computational resources making such big blocks that everybody else learns about its blocks very late, while it already has spent considerable time finding the followup block.

The uneven propagation is due to both network latencies and verification times of the block that is being propagated, so this is a compound problem from both slow verifiers and high latency/low bandwidth.

For more detail see Information Propagation in the Bitcoin Network (disclosure: I am the author of that paper) and Majority is not Enough: Bitcoin Mining is Vulnerable.


Why is the block size is not filled with transactions? Since a miner can acquire the fees from the transactions in a block, the miner should package as many transactions as possible in a block.
However, I recently found a block (blocknum: 602297) with a relatively smaller size, whose size is only 134 kB.

More details about the block can be found in block: 602297.

Why does it not contain more transactions to earn more fees?

Miners pick transactions from the mempool which is the queue of unconfirmed transactions. When there are fewer transactions waiting than would fit into a block, the block will not be full. The miner could create more transactions themselves, but that would be only useful if they wanted to send one themselves already in the first place, otherwise they'd just be paying themselves to include useless data.

So, why are so few transactions waiting to be confirmed sometimes? The traffic in the Bitcoin network appears to have both a daily and a weekly cycle. It tends to start picking up when Europe's business day starts, and slows down a bit when the US West Coast goes home from work. Weekends tend to have less traffic altogether. Of course that is overlaid by greater trends in Bitcoin, so demand will surge during price rallies for example.

Mempool Visualization of last seven days via jochen-hoenicke.de

Additionally, the block production is probabilistic. Blocks come in random intervals roughly ten minutes in average, but that'll mean that sometimes there may be three blocks in one minute and the queue gets flushed, or no block for an hour and the queue builds.

UTXO Set Size over past year via statoshi.info

In the end, network participants should ensure that the queue doesn't flush completely. When there is little demand for blockspace, users could be consolidating their tiny pieces of Bitcoin: the unspent transaction output set grew from 50M to 64M in the past year. In a perfect world, we'd always have a backlog of consolidation transactions that we could chew through when no other transactions are waiting.


What is the NTX ID that Mt. Gox refer to in their Feb 17th Statement? Mt. Gox have released a statement concerning their recent withdrawal problems.

  In order to launch the new system, we are going through the following
  steps:

  Re-indexing the entire Blockchain (approx. 32 million entries)
  Fully deploying the new NTX ID
  Implementing a new bitcoin withdrawal queue that needs to be tested

Is the NTX ID something native to the Blockchain.info API? Does Blockchain.info host it's own separate version of the actual blockchain with additional meta data?

This is closely tied to the transaction-malleability bug, which allowed the re-posting of unconfirmed transactions with an altered transaction id. The issue increased the complexity for bookkeeping for services with a high number of transactions. Especially, some people used it to sneak duplicate withdrawals from Mt.Gox.

The change to the transaction ID was made by pushing additional zeros to the signature's front. I think NTX ID stands for normalized transaction id which will map all duplicate transactions back to the same representation, even when they were altered by exploiting the transaction malleability.

It is an internal blockchain representation used by Mt.Gox and has nothing to do with the service blockchain.info.


What would a full node do if it finds an error? Full Nodes are supposed to validate every transaction and block in the blockchain, and provide valuable links for distributing blocks.  And because the consensus algorithm is extremely good, there are no invalid transactions in the block chain.

But if an invalid transaction was somehow created on a mined block, and identified by a full node, how would the node respond?  Would it show an error to the local user?  Distribute some sort of warning or notice on the network?  Simply crash?

From a local user's point of view, the most important function of the full node is to tell them when funds are received and confirmed.  For instance, if the local user is a merchant, this is what they need to know in determining when to ship goods to customers.

So from this perspective, the most important thing for a full node to do if it finds an invalid transaction or block is to ignore it.  For an invalid transaction, it needs to not report the transaction to the user as received funds.  For an invalid block, it needs to not report this as confirmation for the transactions in the block, nor as an additional confirmation for transactions confirmed in previous blocks.

A merchant considers an invalid transaction to be the same as no transaction at all.  A user who paid with an invalid transaction effectively just hasn't paid, and will be ignored until they come up with real funds.  An invalid block simply isn't a confirmation, and if we are waiting for some number of confirmations before shipping goods, then we just have to keep waiting.

Sure, the node can ban the other node that sent the offending data, or log an error, or whatever.  This is a good idea as far as saving bandwidth for valid data, or giving the user some general info about the state of network, or aiding in debugging.  But all this is secondary.


Curl bitcoin protocol for learning? I'm trying to understand bitcoin RPC Peer to Peer protocol to implement SPV on a web-application that should receive crypto payments.

The problem is that on the examples I found, examples are about RPC requests to a full node I'm supposed to own, but I want to query other nodes (nodes I have no user:password for).

As far as I understand, bitcoinj and full wallets connects to the network make requests to them without user:pass, but I could not found how that requests are built, and how can I test and try them using curl to learn how it works.

Is there a api documentation I can check and a list of nodes I can query?

Also, I understand that to establish a connection I should send the version request, but from curl I don't have a "version".

I've found this documentation about the protocol, but I've found no way to test that out in simple terms (like curl), to know what is being sent-received.

I want to understand what queries I can make using this protocol without running a full node.

update

It seems that what I wanted to know if it is possible to learn about the bitcoin protocol making test calls with curl to full-nodes (not RPC). It seems that it is not possible.

Nodes communicate to each other using the peer-to-peer protocol, typically using TCP port 8333. They rumor blocks and transactions to each other, request data, tell each other about other IP addresses in the network, ...

Each node implements the level of validation they desire; some download all blocks and validate everything (full nodes), some do that but don't keep the blocks on disk after validation (pruned full nodes), and some download only the transactions they're interested in along with proofs they wee including in the chain without verifying validity.

Independently, some node software also exposes an interface to communicate with users or other software using an RPC protocol. This interface however is not public for good reasons:

It's usually not designed to work in adverserial settings (for example: the RPC interface may expose a command to shut down a node, or dump its private keys, ...).
It's not hardened against denial-of-service (DoS) attacks.
It's not others than the node's operator should be using anyway, as the information provided by RPC is an interpretation of what is going on in the network; it's not verifiable. So if exposed to untrusted party, node software could just lie, possibly defrauding the party using it.


How to sweep a Bitcoin Core wallet? I have created an account in a bitcoind wallet which has got some 1.055btc which has to be wiped out to 0.00BTC to another account.  so the actual BTC has to be sent is 1.055 - transaction fee.

Is there any calculation for bitcoin to find transaction fee or any jsonrpc to find estimation fee like ethereum (balance - (gasprice*gaslimit)).

With recent versions of bitcoind, you can use sendtoaddress with the optional subtractfeefromamount argument.

sendtoaddress 123abc... 1.055 "" "" true

Creates a transaction with 1.055 BTC of inputs.  The destination address will receive a little less than 1.055 BTC, after an appropriate fee is subtracted.


Attach and persist identity to address With Bitcoin, you can sign messages to prove that you are in possession of a private key associated to a public key. Correct me if I'm wrong, but from my understanding this only works with a separate communication channel, such as email or a central system where you request the other system participant to sign the message for you.

I think this works great if you want to confirm a shipping address in order to make sure the person who paid is aware of where the shipment goes (if you want to accept the fact that everyone in the network can see your shipping address).

But does that work for a notary use case, for example? How can you know, using Bitcoin message signing, that the company "AMCE Ltd." is actually behind the address "XYZ"? Is this only possible if you actually call AMCE Ltd. or email them for them to sign a message for you? What if you can not trust the communication channel (e.g. message gets intercepted by a potential attacker).

The only thing I can think of that can solve the problem is introduce a middlemen you are required to trust (from what I understand, Onename.com is such a middleman) and who does the background check for you. But from my understanding, this would eliminate a significant part of the value of Blockchain. Why would you need a distributed ledger anyways if you already trust the middleman?

I think this works great if you want to confirm a shipping address in order to make sure the person who paid is aware of where the shipment goes (if you want to accept the fact that everyone in the network can see your shipping address).

You can embed an shipping address into the blockchain using OP_RETURN or something similar, but a much simpler way is to sign your shipping address using your bitcoin address, and send that via a private channel.

  But does that work for a notary use case, for example? How can you know, using Bitcoin message signing, that the company "AMCE Ltd." is actually behind the address "XYZ"? Is this only possible if you actually call AMCE Ltd. or email them for them to sign a message for you? What if you can not trust the communication channel (e.g. message gets intercepted by a potential attacker).

You need some sort of trusted communication channel to send an address. If you do not have one, most methods of starting one (like SSL) involve some kind of trusted third party.

But that's not unique to Bitcoin, of course. There's a huge problem with fake, nonexistant businesses on Google Maps. In order to create a naming system that resists tampering and fraud, you must have 1) some system to detect and remove fakes, and that system must work consistently and 2) some way of punishing people who create fakes.

  Why would you need a distributed ledger anyways if you already trust the middleman?

One reason is that there are different kinds of trust in play. For example, I trust Comodo to verify SSL certificates before signing them, but I don't trust them to hold any of my personal data. I trust my bank to route payments to the correct recipient, but I don't trust them to not charge me an inactivity fee.


How are transactions in discarded forks merged back into the blockchain? It sounds like transactions in discarded blocks are propagated back to the network to be included in the next block to be generated, is that the case?  If so, is that triggered in the client when it discards a fork and accepts the longer chain?

I see some info and discussion on this in the protocol and forums, but it's not entirely clear, and sounds like the heuristic is somewhat complex and non-deterministic (eg, some transactions can just be lost).

The client will import all the lost transactions into its pool of transactions it's trying to commit. Miners will include those transactions into blocks mined for the new chain, if they're not already in it.

Since a miner won't keep a conflicting transaction in memory anyway, it's not possible for the client to cause a conflict by accepting the transactions in the blocks that are undone. Miners should immediately accept all transactions that meet its transaction fee policy and fit as many of them as they can, in priority/dependency order, into blocks they mine.

Of course, coinbase transactions (the payment for mining a block) are forever lost. This is the primary reason that many more confirmations are typically required before coins from these transactions can be spent.


How to calculate signature generation, signature verification, transaction verification speed? I read this post on Bitcoin Scalability.
As far as I understand sequence of steps in Bitcoin, simplified version of performing transaction should be following:

On Sender's side:

Create transaction
Generate signature and sign
Broadcast the transaction to peers

On Peer's side:

Get the transaction
Verify signature
Verify transaction
Pass to the next peer. - ???

//if a peer is a miner

Collect transactions into block
Calculate PoW for the block
If found include it to his version of the blockchain
Broadcast the block (with nonce and targe)

Other peers:

Get the block
Verify that PoW for block is Valid
Include the block into their version of the blockchain

I would like to understand Bitcoin's performance speed at every step in general or given my specifications of the laptop if possible.
How I can calculate signature generation, signature verification, transaction verification, block PoW verification, **etc speed?**

All of these things you want to calculate are highly dependent not just on the hardware you use, but the software you use. For example you can generate signatures many different ways and every bitcoin wallet generates them a little differently. The signature generation performance of Java in bitcoinJ is going to different than that of Go in btcwallet is going to be different than that of C++ in bitcoin core and so on. In addition, deterministic signatures are going to have a different performance profile than non-deterministic signatures and so on.

That said, if you really wanted to, you can run some performance testing on any one of these, though not all of them exist. For verifying signatures in btcd, there is a benchmark test that you can use.

Note that block verification (or creation) necessarily means verification of all transactions in the block as well as checking the proof-of-work. Checking the proof-of-work, by the way, is really as simple as a calculation of a double-sha256, or a single "hash" that mining gear can do billions or trillions of per second.


Move BC before/after reformatting I plan on reformatting my main drive. I did this once before and of course lost access to my Bitcoin Core wallet and the Bitcoins therein.

Does anybody know how I can go about transferring my Bitcoins before or after I reformat my drive?

Can I just copy and paste the Bitcoin Core folder to my second drive before formatting, and then transfer it back afterwards?

You can just copy your wallet.dat from your local wallet directory. If using windows it will be in %appdata%/Bitcoin. After reformatting download a fresh copy of Bitcoin Core and replace the wallet.dat with the one you saved prior to reformatting.

There's also the method of using the backupwallet and importwallet commands. https://bitcoin.org/en/developer-reference#backupwallet


Setup bitcoin-seeder I wanna setup a dnsseed for a crypto currency using sipa/bitcoin-seeder on my VPS.
The server is hosted at NetCup and running the software as root, and also listening on port :53 UDP.

My domain is hosted at GoDaddy and got those records:

dnsseed.machinenode.de  1800     IN     NS  ns01.machinenode.de.
ns01    1800     IN     A   SERVER_IP

However, if I use dig -t NS dnsseed.machinenode.de I just get NXDOMAIN as status and no namserver returned.

Is anyone familar with setting up a dnsseed and able to help?

Thanks and friendly regards

Nico

EDIT:

After switching my servers OS from CentOS 7 to Ubuntu 16.04 things started working.

When you add or alter a DNS record you need to update the serial number in the zone data and take into account the TTL (time to live) which is used by clients as a cache refresh interval.

If you are using a hosting provider's web-based control-panel, it probably takes care of the serial number for you but it won't adjust the TTL in advance of the change. Therefore your change can take a while to propagate through third-party and client DNS caches.

NXDOMAIN means "no such domain" - so far as the nearest DNS server is concerned, it has a still-valid set of data for the parent zone and that does not contain the delegation you enquired about.

$ dig +nocmd +multiline +noall +answer machinenode.de
machinenode.de.         600 IN A 50.63.202.40

$ dig +nocmd +multiline +noall +answer dnsseed.machinenode.de
dnsseed.machinenode.de. 30 IN A 92.242.132.15

Those numbers 600 and 30 are the TTL values.


Is it still possible to issue your own assets on Ripple? I was talking to some folks interested in facilitating trade of non-monetary assets (power to be specific). I was thinking that Ripple as a platform may be more suited for such trades than Bitcoin. However, when I was researching Ripple today, it seemed to me that their marketing is now almost exclusively focusing on banks.

Is it still possible to issue your own assets on Ripple? Do you need to be a Ripple gateway to do so? Is there another solution to account for self-issued assets in high frequency with an non-fixed group of participants cheaply?

It is definitely still possible to issue your own assets. Here is the process:

Create an account on the Ripple network, you will need to get someone to send you enough XRP to perform transactions, $1 worth should easily do it.
Submit a transaction to set the DefaultRipple flag on the account. If you don't do this, people who hold your asset will not be able to send the asset to each other.
Choose a three-character code for your asset like PWR.
Convince other people to set up trust lines to allow them to hold your asset.
To issue your asset to people just use a Payment transaction.

Your asset will have the following characteristics:

Only you can issue it.
Anyone who agrees to hold it by creating an appropriate trust line can receive it.
People can place offers to trade your asset for any other asset tradeable on the ledger.
Your asset can be used as a source asset or destination asset for payments with automatic pathfinding so long as there are sufficient offers in the ledger.

The question of whether you need to be a Ripple gateway may reflect a misunderstanding about what a gateway actually is. A "gateway" is just a company that offers the service of issuing and redeeming a digital asset on demand. What makes you a gateway is that you bridge some asset outside the ledger to a digital asset on the ledger.

If you do this, you're a gateway. Whether or not this custom asset requires a gateway depends on how you are expecting people to be able to use it.

For example, if you wanted a custom asset that represented gold, you would probably need some way for people to trade the asset for actual gold (or, at least, the current value of that gold). And you would probably need some way for people to get the asset in exchange for actual gold (or, at least, the current value of gold). That would be the gateway's role -- to bridge the digital asset to the external value it's supposed to have.

By contrast, if you want an asset that represented an hour's worth of work, you might not need any gateway at all. You might let anyone issue the asset in exchange for an hour of work with it being redeemable against their own work. This wouldn't require a gateway, but you'd be trusting individual issuers to honor their obligations. (And it wouldn't really be one asset, since Jeff owing you an hour of work would be a slightly different asset from Sarah owing you an hour of work.)


Is Primecoin mining competitive? Meaning, if more people mine Primecoin, does it get harder to mine? From what I understand, it shouldn't, but I want to be sure.

Primecoin mining is doubly competitive (in the sense you described). In fact, the amount you will mine with given power is inversely proportional to the cube of the total network compute power.

As in Bitcoin, a difficulty parameter adjusts based on the mining rate to achieve a desired block frequency. Not only that, but the mint rate is inversely proportional to the (square of the) difficulty. Unlike Bitcoin which has a fixed schedule, which at worst can be slightly accelerated with increasing mining power, in Primecoin the total number of coins that will ever be generated will decrease with increasing competition.


Why raw Bitcoin transaction are in hex format? Why are Bitcoin raw transactions in hexadecimal format? I am looking for scientific articles / books on this topic and I can't find anything, so my question is: why are they in hexadecimal and can there be any article on this topic?

Because unlike addresses, hex raw transactions aren't meant to be seen or used by end-users. Hex is easy to encode and decode (every two character in hex represents one byte*, whereas in base64 one character is 3/4 byte and Base58 is log(58)/log(256) = 0.7322476244 bytes) so it is convenient if you want to debug your implementation.

*: which means if you can find an erroneous byte in the transaction that you can CTRL+F for that byte in the serialized hex transaction and change it since unlike Base58 or Base64 an encoded character depends on the part of only one byte and never two neighboring bytes. Hex is the highest base with that property. Binary (1 bit divides 8 bits), Quaternary (2 bits divide 8 bits), not octal (3 bits don't divide 8 bits), and hexadecimal (4 bits is the largest divisor of 8 that isn't 8) are other bases with that property, so hex is the most efficient one among them.

Or simply, it was created as an internal tool which didn't need the complexity of a new format that provides space efficiency or avoiding similar characters (or checksum, even though hex doesn't depend on checksum) that is user-friendly.

From Bitcoin QT 0.7.0 release notes:

  Added raw transaction API. See https://gist.github.com/2839617

Gavin Andresen, who created and submitted the hex transaction encoding code to Bitcoin Core/QT wrote:

  I considered parsing/writing BIP 10 format for raw transactions, but
  decided that reading/writing BIP 10 format should happen at a higher
  level and not in the low-level RPC calls. So 'raw transactions' are
  simply hex-encoded into JSON strings, and encoding/decoding them is
  just a couple of lines of already-written-and-debugged code.


Is there a good guide to convert a Bitcoin Core Wallet to Bitcoin Cash (on OSX) I got a wallet in Bitcoin Core.app version 0.14.2 on OSX. My last transaction is from years ago. How can I convert my wallet to Bitcoin Cash and what are the do's and don'ts? Basically I don't want to loose my coins. I heard that I could claim both new coins, its that true and how to do so?

The first paragraph in the answer to this question applies in your case (possible duplicate question?). As you run a full client (Bitcoin Core) it's easy, as Bitcoin Cash has equivalent apps which are forks of this. I used Bitcoin ABC.

I used a separate machine for the Bitcoin Cash wallet. It may be possible to run both on the same machine (with different data directories - see this issue) but despite being called 'Bitcoin ABC.app' if you try and install it on the same machine as one with Bitcoin Core, you will get a message offering to overwrite the latter. Comments on how to accomplish this on one machine are welcome.

Backup Bitcoin Core wallet (~/Library/Application Support/Bitcoin/wallet.dat) to a USB stick (we'll transfer it later). This is the key do, which of course you will have done already
On the second machine download Bitcoin ABC from bitcoinabc.org. Currently it is the .dmg file in the osx directory. I expect they'll improve the UX to have an easy Mac-specific download button soon.
To get the blockchain to sync either just run Bitcoin ABC until it does (it'll download a chain similar in size to the Bitcoin Core one i.e. - 150GB+) or copy the legacy part of the chain (common to both coins) from the Bitcoin Core data directory. As OP comments, a Time Machine or other backup is perfect for this. I'll update this answer if someone can comment on exactly how to do it without one, as ideally you'll need most of the contents of the blocks and chainstate directories - up until just before the fork date; 1 August. You'd then only need to sync since then.
Overwrite the new wallet.dat file Bitcoin ABC created (same location as above) with the one from step 1.
With both apps synced you should now be able to send/receive BTC/BCC independently of each other from the relevant app - i.e you now own both. Should you come down on one side of the fence & sell all of either you can of course remove it's app, data directory & return the spare/borrowed machine.

In terms of dont's, you should ignore any advice to export/mess with your private keys - especially is someone offers to claim your BCC coins for you if you send your keys..


Does the official Bitcoin version 0.8. client still download the whole blockchain? The release notes for version 0.8 of the "Satoshi" client say

  "Bloom filter" support in the network protocol for sending only relevant transactions to lightweight clients.

Is this an optional feature or on by default? Does this mean that the client will no longer download the whole blockchain? What is a relevant transaction? Is it only transactions related to keys in my wallet?
Does that mean that I can no longer keep the machine that downloads the blockchain separate from the machine that hosts my wallet (and that I keep offline, copying over the blockchain manually by USB). If so, is there an option to get the old behaviour?

0.8 doesn't actually use the bloom filter feature, but it enables support for letting other nodes use it.

0.8 does download the entire chain.


Unconfirmed transaction for 10 days. Please help. Unconfirmed transaction from my wallet. Here are the details.
https://blockchain.info/de/tx/2b91804cada49e52774d3c4fd3c02e3932c03b358c8ea4d74abc4d407d5f429c

How can I increase the transaction fee or what do I need to do to get the transaction through?

Many thanks for your help!

Your listed fee is 44 sat/B.

From this page, you can probably expect anywhere from 30 minutes to 12 hours at that rate.

Since your transaction was submitted more than 10 days ago, I wonder if there is some other reason it isn't getting confirmed?


What does event_base_dispatch do? Source code (src/httpserver.cpp):

static void ThreadHTTP(struct event_base* base, struct evhttp* http)
{
    RenameThread("bitcoin-http");
    LogPrint("http", "Entering http event loop\n");
    event_base_dispatch(base);
    // Event loop will be interrupted by InterruptHTTPServer()
    LogPrint("http", "Exited http event loop\n");
}

Can anyone help me understand this portion of logic? In particular

What is the HTTP magic happening here event_base_dispatch(base)?
Why are we renaming threads?

event_base_dispatch is part of libevent, a library that takes events from network sockets, and uses them to call callbacks in your program. To find out what it's calling, you should look at the struct event_base* base variable.

bool InitHTTPServer()
{
    // [...]
    base = event_base_new(); // XXX RAII
    // [...]
    http = evhttp_new(base); // XXX RAII
    // [...]
    evhttp_set_gencb(http, http_request_cb, NULL);

There are three important calls here. event_base_new creates the struct that tells event_base_dispatch how to handle incoming connections. evhttp_new tells libevent to interpret incoming data as the HTTP protocol, instead of passing along raw bytes. evhttp_set_gencb tells libevent which function to call when new requests come in. This is where that ends:

/** HTTP request callback */
static void http_request_cb(struct evhttp_request* req, void* arg)

  Why are we renaming threads?

Debuggging and performance logging.


Network of 3rd-parties to monitor for Lightning Network theft transactions Whenever the balance of a Lightning channel is updated, the counterparties must revoke the Commitment Transaction reflecting the channel's earlier balance, and sign a new Commitment Transaction with the new balance.  If the counterparty attempts to steal funds by broadcasting the revoked transaction to the Bitcoin network, then the other party can broadcast an anti-theft transaction that forfeit's the counterparty's funds.

A common objection I hear from skeptics of the Lightning Network is that, as a consequence of this, you have to be constantly online and monitoring the blockchain to prevent this kind of fraud.

However, as mentioned in this post, I've heard it proposed a number of times that:

  the anti-cheat transactions can be safely distributed in the network
  and have an attached bounty to make it interesting for third parties
  to monitor on your behalf. When a third party broadcasts the
  anti-cheat they'd assign the bounty to themselves, but would be unable
  to touch the remaining funds.

My question is, has this ability to broadcast anti-theft transactions to third-parties interested in monitoring the blockchain for theft transactions actually been implemented somehow?  Do the existing Lightning implementations include a protocol for broadcasting anti-theft transactions to third-parties?

I've asked the Lightning developers on Slack and this was their response:

Please note that things might change in the future.


What does nonce mean? I'm trying to learn how litecoins work (and learn new programming language at same time by writing a crappy little miner).

I'm stuck with getdata. Here http://litecoin.info/wiki/Scrypt
it says that data is 160 characters long. But its not. Its quite a bit longer than that.

So my questions are:

1.) What is nonce and how do I use it?
2.) What is bits?
3.) What are all the 0's after the bits.

EDIT: Ok, I think I have to use just first 160 characters of data and ignore the rest. Correct?

A nonce is just a value that you keep changing in the hope that after you hash all the data the hash will be less than the target value. See https://en.bitcoin.it/wiki/Block_hashing_algorithm (litecoin is similar only uses scrypt instead of sha256 hash function.)


Why are miners allowed to not announce their mined blocks to the network? My question relates to a 51% Attack. As far as I understand, for this to work the miner must have a secret fork of the main blockchain, which he will later announce, whenever it becomes longer than the main blockchain, thus reverting a previous transaction and permitting double-spending.

So then, why not just force blocks into the main chain the moment they are mined? What is the rationale for these private forks?

"Why are miners allowed to not announce their mined blocks to the network?" you can not prohibit to mine blocks and not announce them. if you do some mining-calculations on your mining-hardware, you can do that offline. no one knows that until you publish your block(s).

"So then, why not just force blocks into the main chain the moment they are mined?" you can not know assuredly the timestamp when a block was mined because the miner can write an arbitrary timestamp in the block. furthermore all bitcoin-nodes/-miners should/will follow the longest block-chain by definition. the timestamps in the blocks does not really matter.

this "private mining" and resulting doublespend-attacks are absolutely possible. the bitcoin community trust that no one ever will have enough hashing-power to do this attack.


Create Raw Transaction in BitcoinJ offline starting from UTXO info I would to create offline Raw Transaction (without signing it) in BitcoinJ. The information I know about the inout is UTXO's hash, index, amount, address, and script. I also know the receiver address of the final transaction, the change address, the amount and the fee.
I'm fighting with the TransactionOutput constructors to build the input for my transaction. I don't know where to set UTXO hash and index - i think in the "parent" parameter, but i'm failing. This is my snippet:
Transaction parent = ???
Coin value = Coin.valueOf(satoshi(utxo.getAmount()));
Address to = Address.fromBase58(params, utxo.getOwnerAddress());
//where do i put utxo hash and index?
TransactionOutput input = new TransactionOutput(params, parent, value, to);
tx.addInput(input)

I'm fighting with the TransactionOutput constructors to build the input for my transaction. I don't know where to set UTXO hash and index

I think you can build the input based on a previous transaction output (outpoint) as follows:
byte[] scriptBytes = {};
Sha256Hash vinTxId = Sha256Hash.wrap(txId);
TransactionOutPoint outpoint = new TransactionOutPoint(params, index, vinTxId);
Coin value = null;
TransactionInput input = new TransactionInput(params, transaction, scriptBytes, outpoint, value);
transaction.addInput(input);

Not sure if this is the best ways to do it.
EDIT: This does the same as the code above.
transaction.addInput(Sha256Hash.wrap(txId), index, ScriptBuilder.createEmpty());


listtransactions via RPC returns false in PHP In PHP I run

$bitcoin->listtransactions("count = 5, offset = 0");

In the hope of getting 5 transactions only, but I get the response

bool(false)

When I run $bitcoin->listtransactions(); alone, it works fine.

What am I doing wrong?

According to documentation, there are several optional parameters you can provide  in order to retrieve list of transactions for account(s).

Function:

listtransactions [account] [count=10] [skip=0] [include_watch_only]

[account] (string) leave "" for default account and "*" for all accounts, default ""
[count] (int) number of recent tx to return, default is 10
[skip] (int) number of recent tx not to return, default is 0
[include_watch_only] (bool) include watch-only addresses, default is false

I suggest to try the call with all 4 parameters:

$bitcoin->listtransactions("*", 10, 0, false);

I am not PHP expert but the output you get might be indication of missing parameters.


Is it possible to mine Peercoin and Litecoin simultaneously on the same machine? Using GPU(s) for Litecoin (scrypt, Cudaminer) and a USB ASIC for Peercoin (SHA-256).

OK, I have this set up now. BFGMiner is running the USB ASIC for Peercoin and cgminer is running the GPU for Litecoin. I also have cudaminer running my second GPU for Litecoin, which was necessary as cgminer didn't like running two different GPUs at once. All works nicely together now.


How many confirmations does bitpay wait? I've made one purchase with bitpay, it has been received by the merchant who is stating "Payment received. Waiting for more confirmations". While this is fine, I find it quite odd I didn't manage to find exactly how many confirmations are they going to wait for.

Is it configured merchant-side, or is a constant for all bitpay transactions? If the latter, how many?

Really quite puzzled I didn't find anything about that in the bitpay site...

The number of Bitcoin network confirmations required for a particular transaction to be considered complete is determined by the BitPay account holder on a transaction-by-transaction basis.

Entities set the Risk/Speed of a transaction to "Low" (6 confirmations), "Medium" (3 confirmations), or "high" (1 confirmation) via the API when an invoice is created. The entity's default Risk/Speed setting for new transactions can also be changed in the merchant's settings after logging in.

For clarity, due to the way in which the Bitcoin network secures transactions, "speed" correlates directly with "risk" in that the entity is taking a (relatively) high risk by only waiting for a single confirmation before releasing the product or service. The entity can decrease its risk that the payment will become invalid by waiting for further confirmations that the rest of the network accepts the transaction. This is a choice the bitcoin acceptor must make, weighing the risk of losing the payment vs keeping the buyer happy (by giving him/her the product faster).


Why are there always some unconfirmed transactions? As far as I know, a miner begins creating a new block as soon as it receives a valid block from another miner. It then begins to construct and hash the next block on top of the block it just received by using all the transactions it has in its memory pool.

However, there are still some transactions that are not easily confirmed (or not confirmed at all), in the BitCoin.

Can anyone explain to me why this can happen? Is it because of some transaction selection mechanism that the miners use? or is it because of the unreliability of the connecitivity between the nodes? or something else?

Transactions selected in mining blocks is not like a queue, where you wait your time and get to the front eventually.

Fees are not mandatory, but should be included in every transaction, if you have sent none, or low fees, you can still be selected (although it's highly unlikely), but the miner would be silly as they could just take the higher fee offered by someone else. Remember when you post a transaction you offer a fee.

If you owned a bakery and made 1000 loaves of bread a day, but had 1500 customers, would you sell them to someone offering $0.10 when other people are offering $1.00+.


How can I recover BCH sent to a Trezor BTC address? I accidentally sent my BCH from Breadwallet to a BTC address from my Trezor wallet. Is there any way to retrieve that back to my BCH wallet? The BTC wallet I sent the money to is mine, so I own the keys but the address starts with 3.

Following the steps from "How to claim your Bitcoin Cash (Bcash)" by Trezor Blog should do the trick.

https://blog.trezor.io/claim-bcash-bitcoin-cash-bch-bcc-trezor-wallet-f0a810d5864a

Steps:

1) Make sure Trezor uses latest firmware

2) Connect your Trezor

3) Go to Coin splitting tool


BIP65: "replacing ntimelock entirely"? BIP65 discusses replacing the ntimelock field with the new CHECKLOCKTIMEVERIFY OP code:

  As an aside, note how if the SignatureHash() algorithm could optionally cover part of the scriptSig the signature could require that the scriptSig contain CHECKLOCKTIMEVERIFY opcodes, and additionally, require that they be executed. (the CODESEPARATOR opcode came very close to making this possible in v0.1 of Bitcoin) This per-signature capability could replace the per-transaction nLockTime field entirely as a valid signature would now be the proof that a transaction output can be spent.

How is this different from the functionality of ntimelock? (I.e. Why would a signed transaction using ntimelock not be sufficient?)

nLockTime affects transaction validity only, if you make a transaction with an nLockTime in the future (by block height or unix time) it is invalid until that time has been passed. This means that if you sign a locked transaction, you can also just as easily sign a different transaction spending the same outputs that is spendable today.

This is useful if you wanted to make a contingency transaction and give it to a third party which spends the output to a charity if you aren't around to make use of it in a years time. If you are around you can spend it in an alternate way, if you aren't a third party can broadcast the locked transaction to the network once it has become valid.
The limitation is in that you can't prove that funds are actually locked in this state, there's no way of knowing that the person won't just spend around an nLockTime transaction in a different way and bypass the restriction on when they could spend the funds. Setups are possible where a user sends an output to a new key, makes a locked transaction to themselves at another address, then destroys the private key, but this is incredibly flimsy and has no real assurances that the key wasn't retained secretly.

CHECKLOCKTIMEVERIFY is a new proposed opcode that allows you to use the current real time in a transaction script, allowing some more interesting and provable use cases that involve actually locking money rather than a particular transaction. This is closer behavior to how people normally think nLockTime should operate when they first hear about it.

With CLTV, scripts that have different preconditions for spending based on the time are possible. Rather than limiting to a single presigned nLockTime transaction, a person could have every transaction in their wallet spendable by a specific charity key if 2 years pass without them being spent, or a 2-of-3 multisignature could have a contingency backdoor where an additional party can be included if the funds have not been used for an appreciable amount of time.
CLTV also allows for payment channels to be used even with transaction malleability. A mutated transaction can break nLockTime refund transactions at any point, which makes them quite risky at present. With CLTV the payment channel transactions can just timeout and be refunded to the creator without the need for any further interaction.

BIP65 is potentially dangerous to end users as there are pretty much no take backs (if you lock your money for 5 years, there's no back doors or workarounds that will get your money back before then), but that said, that's a user interface problem rather than a fundamental security one. There's little contention about soft forking this change into the Bitcoin codebase in the future.


Single signature P2SH redeem script How would one go about creating a single signature P2SH transaction? (Analogous to a "2 of 3 multisig P2SH" but instead I'm asking about a "1 of 1".)  Would one wrap a standard P2PKH scriptPubKey inside of a P2SH redeem script? Or is there a better/preferred way of achieving this? Thank you.

First of all, you can use 1-of-1 msig for a redeem script:

OP_1 push(pubkey) OP_1 OP_CHECKMULTISIG

But this one is better:

push(pubkey) OP_CHECKSIG

And of course, you are welcome to use segwit form

OP_0 push(hash160)


My balance gets sent back to me I'm using the blockchain API to make a "send to many transaction". In the recipients I only put other people addresses, but somehow the API sends the remainder of my balance back to me.

For example:

  https://blockchain.info/tx/b4c3203815fdf61616c0a2358fa0388e734bf53503f0a659d3c9fc7863da8adc

I had 0.01 btc in my wallet. I sent 0.0000543 BTC to two different recipients, but somehow my whole balance got spent and my 0.0097914 BTC got sent back to me. I didn't ask for this.

EDIT: It also happens on a singular transaction:

  https://blockchain.info/tx/2eb4273313e27c2ede86844c58f687dc535aeaa5207e9a9c4a526a032cd5143a

I've inspected the API calls and they are correct. My address is no where to be seen in it.

Why is this happening?

That's how Bitcoin works. Accounts don't really have balances, they have unspent outputs. You have to spend entire outputs, but you can return amounts to yourself to create a new "change" output that you can spend later.