does Bitcoin Core automatically use reindex when you specify txindex=1 with existing sync'd chain? I sync'd a bitcoin node with Bitcoin Core Qt GUI. Hundreds of GB, naturally.
I realized I needed txindex on for Lightning node stuff, so I changed the conf file via the GUI to include txindex=1. I launched it on my Mac and then slapped myself in the face - how would I tell it to reindex instead of throwing away my database?? Didn't wanna download all those GB again.
But then I noticed debug.log started saying this:
2020-11-30T06:07:43Z Syncing txindex with block chain from height 498564
2020-11-30T06:08:28Z Syncing txindex with block chain from height 500148

which, if I remember right means it's doing a reindex, not a full-blown restart of a sync. Checked on disk, and indeed, files still there.
I'm grateful, but also mystified. Questions:

does Bitcoin Core see when you're about to throw away a full sync with txindex=1 and then inject a -reindex into the start command?
how can I see the startup command that bitcoin-qt uses when I start with with the Mac OSX launcher?

Changing the txindex setting no longer requires a reindex since version 0.17.0 (see https://bitcoincore.org/en/releases/0.17.0/, under "Transaction index changes). It'll update just the txindex itself whenever it's out of sync automatically.
So:

It doesn't inject a reindex (which would cause throwing away the entire chainstate database, and rebuilding it). It just detects the txindex is out of date, and scans the unindexed part of the chain in the background.

The command is "bitcoin-qt", almost certainly, with nothing more.


Send BTC with Electrum: which address is used? I'm new to bitcoin and I've downloaded Electrum.
I can see the new wallet generated a few addresses for me.

If I want to receive BTC, I just need to give one of those addresses.

But if I want to send BTC (using the Send tab), I don't see any field to choose which address will send the BTC.

So, how does it work? How will Electrum choose the address to take them from?
Can it be sent from multiple addresses? (e.g. To send 1 BTC, address A send 0.4 and address B send 0.6)
Is it possible to choose a specific address to send the BTC? (e.g. I want specifically address C to send the 1 BTC)

Thank you.

Great question, I've always wondered this too. But im pretty sure that Electrum, by default, will send your oldest unspent outputs first. (Sorry if example is a little repetitive)

E.g. If you recieved 0.4 on address A 3 days ago, 0.2 on address B 2 days ago and 0.1 in address A again 1 day ago, it would send 0.4 from address A, then 0.2 from address B, then 0.1 from address A again.

Im pretty sure you can change this in Dev mode. I'll look into it.

As I said, in order of oldest unspent outputs (balances)
Yes, they can send from multiple addresses. But it will try to send from one address if possible.
Try the coin selection setting in settings.

EDIT - Ah, yes you can get it to spend from certain addresses. Go into your Electrum settings, and set your "Coin Selection" mode to privacy.


Can I create a Blockchain copy (up and running) in order to make my own bitcoins transfers without Fee? I need to make a lot of transfers per day (bitcoins), but the fee is higher than expected. I'm thinking in create a Blockchain copy running on our servers and make the transfers from my wallet and to my wallet by my own blockchain.

I'll charge my self a fee but small and fix.

That can be possible? or a missing something?

Thanks!

Anyone can create their own blockchain, but you will not be able to reconcile your chain with the bitcoin blockchain at a later date (even if you start your chain by forking off from a certain btc blockchain block height).

I don't know what your use case is, but if you're not going to use an already existing blockchain, I would expect that creating a private database (SQL?) would be a better choice than trying to create a private blockchain.

You could, for example, create a database that is designed to mimic the balances of bitcoin addresses you control. Users of your database could freely transfer these 'bitcoins' to eachother on your server, and then only when the user wants to move value to someone outside of your internal system, would you need to initiate an actual bitcoin network transaction.

So the actual bitcoin you control would function as a 'reserve' that your database's value is backed by. This is basically how centralized exchange services work. The users of your system will need to trust you in using it.


What encourages Bitcoin miners to relay blocks? Lets say I'm mining bitcoins, and I've just received a new block that someone else has mined. What encourages me to send that block to anyone else.

I can understand how the original miner wants to send the block around the network (so their block becomes part of the chain) but what incentive do other nodes have to send it around? Wouldn't it be better to not to spread the block, but then build your own block on top of it, in which case you'll then be two blocks ahead and more likely to be added to the chain?

Of course, if everyone does this the result would be terrible, but what incentive is there to encourage people (other than the original miners) to spread blocks besides altruism?

As long as the block reward is much higher than the transaction fees, this is totally not an issue: mining isn't an incremental process, it's a bruteforce probabilistic one.

I.e. in a single moment your chance of solving a block isn't higher if you have been mining that same block for some time: the chance is always the same.

Hence, once a block is relayed to you, you just add it to the blockchain and start immediately to search for a new block on top of that one.

Since the risk of your block being orphaned (currently you'd lose 25BTC ~= 3750€) is many orders of magnitude higher than the cost of relaying it (I'd estimate it a bunch of satoshis), it is always more convenient to relay it.


When manually creating a raw bitcoin tx, how does one determine whether the r or s values require more zeros? Supposedly, if an R or S value will equal a "negative" number, one must append zeros to it and alter the "length" descriptor for the value to reflect that.  When manually creating a raw tx how does one determine whether one's R and or S values require the addition of these extra byte(s)?  All conversions I do with a hex calculator always show positive numbers.

If the first byte is greater than or equal to 0x80 (128), then you need to add a leading zero byte. You can do 0x80 & first_byte. If the result is zero, no padding is needed.


What is the formula for inferring hash rate from difficulty and block frequency? Two parts to this question
1/ There have recently been concerns over drops in hash rate observed on sites such as blockchain.com. However, my understanding is that hash rate is inferred from the difficulty level and the block intervals. I am trying to work out the exact formula for the inference of hash rate.
I know that the average time we can expect to find a block in is calculated with the following formula:
average time to find a block = (difficulty * 32 ** 2)/ hash rate

Would that mean that hash rate is inferred with the following formula?
hash rate = (difficulty * 32 ** 2)/ time interval between the last two blocks

2/ I mainly want the first part answered but if you are feeling rosy today, an answer to this second part would be amazing.
Block times are Poisson distributed. I understand that this allows us to calculate the probability that block times increase to such an extent over the course of a day that it infers a 40% reduction in hash rate.
Does anyone know the exact calculation which would let us calculate this probability?
Here's some rough ideas I have about the calculation:
The following formula allows us to calculate the probability that k events take place in time period t.
P(k in t) = (e ** -lam)*(lam**k / k!)
where lam = (average events which can be expected to be observed per unit of time  * t)

The average events which can be expected to be observed in the case of block intervals is 1 block per ten minutes so 1/10.
Let's say we have hash rate dropping 50% over the course of one day, would that imply that we are observing 288 blocks over the course of 1440 minutes?
If I am thinking about this in the correct way, this would mean the calculation is as follows:
P(288 blocks in 1440 minutes) = (e ** -(144)*((144**288)/288!)

Not sure if this calculation is correct. But to take it further, this would calculate the small probability of exactly 288 blocks being found in 1440 minutes. But if it were possible to calculate the Poisson distribution of block intervals, we may be able to find the probability of finding greater than or equal to 288 blocks in 1440 minutes.
As you can probably tell, my understanding of the second part of the question is limited so if you have an answer to even just the fist part, that would be amazing!

Better Late than Never. I was asking myself the same question...I thought the hash rate of the network came from Full Nodes observation on Miners...I was wrong.
I was looking for this answer on Bitcoin.org, but it's on Blockchain website: https://www.blockchain.com/charts/hash-rate. Look at the methodology at the bottom of the page.
Formula is Hash rate = (Difficulty*2^32)/(intervals in seconds between 2 blocks.
Theoretically you know this to be 600 seconds. So the tell it as H = 2exp32 D/T..
For the second part of your answer, this is from my own inference, but within a day, we can expect a persistent shorter time of resolution per blocks, as hash speed in the long run accelerates at about 400% per year (from technological efficiencies and Mining Nodes being added), meaning about 0,45% per day. Look at Network Difficulty over time, and you see that it continuously increases Difficulty, neutralizing efficiencies, to maintain block resolution time around 10 minutes. If it did not do so, reversibility would increase making reliability of security vulnerable, weakening users confidence, whom would either seek trust signals or defect . On this you can read Nakamoto's introduction of his white paper.
Many factors may affect the slowing of the Networks Hash rate. The most important is not from technology when Hash slips, but from defections (permanent or temporary) of miners associated with Price of the Bitcoin usually.
With blockchain.com latest charts (data from March 18 2021)I can infer the average time of block resolution from the same formula. By doing T = (Difficulty*(2^32)/Network Hash Rate. (21.6TH*4,294967296)/157.6EH = 588.65 seconds or 9'48".
I am new to this, a week or so, and am not and won't be either an investor, nor miner...but I am always very curious, like estimating the number of Miner Nodes, a statistics I cannot find anywhere.
They are about 70 Million lightweight Nodes (mostly wallets), Between 8,000 to 12,000 Full Nodes...but how many Mining Nodes? At Least 1,1 Million to up to 6 Million depending of the formula you create (or I create). If someone has the answer to this, please go ahead.


How is a Ripple gateway different than a conventional bank? Everything I've read so far suggests that a Ripple "gateway" is just like a bank. How is it different than a conventional bank?

Is it possible to have a bank of banks (central bank) in Ripple?

A Ripple gateway acts much like a bank. They compete based on trust, convenience, and cost. They hold other people's money. They promise to handle that money according to their customer's instructions.

One difference is that there's no formal barrier to entry to becoming a Ripple gateway. Someone can open a gateway anywhere in the world and have equal access to the Ripple network for transferring balances, exchanging currencies, and so on.

You can imagine a "central bank" setup on Ripple. One way the central bank could work would be for the central bank to guarantee the performance of gateways and ensure liquidity between them by committing to maintaining a 1-to-1 exchange between the balances of the gateway's it guarantees and its own balances.

Another way a central bank could work on Ripple is if the central bank is the sole currency issuer and manages the account people trust. This would allow a currency with a money supply managed by a central authority.

A bank could also maintain interest bearing deposit accounts on Ripple. A central bank could maintain such accounts for banks or gateways.


Mini-seed of single utxo Let's imagine we have a cold storage wallet that has multiple utxos inside of it and we have a master seed for this wallet.

If we want to spend from coldstorage but want to go first through a new wallet like a coinjoin hot wallet for example we have two alternatives: we could send one single utxo we are interestd in using to that hot wallet or even import the master key to the new wallet (not safe).

It would be very interesting to have a mini-seed of a single utxo from our stash that we could import in the hot wallet and use to spend that single utxo. This way we don't have to create a new tx towards the hot wallet, nor give up information like the whole stash or xpub or subsequent risks of hot wallet.

It would be great to right-click a single utxo on our coldstorage wallet and get the mini seed for that single utxo like another 24 words or 48 words.

Just an idea. Would this be possible?

With existing standards and wallets, no, that is not possible.

Existing standards such as BIP 39 do not directly use the value encoded by the words to produce a BIP 32 HD seed for use in a wallet. Instead they hash the words.

Furthermore, wallets that import that mnemonic do not just use the BIP 32 HD seed from it directly. Instead they use it with derivation paths and derive child keys from it.

If you produced a mnemonic from an individual private key, importing it into any existing wallet would not result in your private key being imported. You would end up with things derived from that private key, but that is not useful to you.

Instead of a mnemonic sentence, what you really want are the Wallet Import Format (WIF) private keys that most wallets can export. WIF is a standard way to encode individual private keys so just use that instead.


Bitcoin transaction query I sent a bitcoin for exchange of etherium from blockchain
But not received anything from the buyer for the past 48 hours.
I am attaching the screenshot of two transactions where the second one is transfer from the my old btc to new btc account showing in green.

The next one is transfer from my new btc to seller account showing in red
see the vallets
1HZCArUVZJCKKwmRVQWh78v3vHZJ2bGPcY   is having nil balance
3HuQXaxerUujQw5kDbSyQLGERVTNvTBnn    is having balance of 0.0049

Is it possible to recover my money ?  Please advise

Is it possible to recover my money?

No, Bitcoin transactions are irreversible.

You could try to contact the other party and either ask about the delay or request a refund.

If you believe the other party is committing a fraud, you can report the problem to your local police - but for small amounts it is unlikely that any action can be taken.


Requirements of own cryptocurrency? What are the requirements and technologies I need for creating my own cryptocurrency?

How to assign value for my own coin?

For example:

1 bitcoin  = 279627.91 INR,
1 Litecoin = 2863.13 INR

You will need a lot of experience and knowledge in many areas including cryptography, networking, programming, economics, and a lot of research into cryptocurrencies themselves before you'll be able to make one yourself.

Values are not assigned by the developers, just by the market, it is supply and demand which raises and lowers the price.


Can you have a cryptocurrency with a single source of truth? Can you have a cryptocurrency that's based on TRUST? (how dare I use this dirty word haha)
Everyone who participates in the system agrees to trust this 1 party only, because they find them trustworthy. And only they are allowed to add blocks to the blockchain. Or they can just process transaction on a 1st come 1st serve basis and eliminate the need for blocks, just timestamp each transaction :)
It would make things way easier but it will make the cryptocurrency centralised?
Is that even legal to do? Can someone create their own centralised cryptocurrency and let people buy/use it?

Is that even legal to do? Can someone create their own centralised cryptocurrency and let people buy/use it?

What is legal/not-legal can change drastically depending on what jurisdiction you live within. If you have legal concerns, you should talk to a lawyer.
Generally though, there is a history of powerful governments shutting down centralized digital-money systems (eg Liberty Reserve), which culminated in the invention of the Bitcoin network (which has no central point of control/authority which can be shut down). This is similar to the evolution of file sharing: tools that were still at least partially centralized were shut down over time (eg Napster), until eventually someone invented a tool that couldn't be shut down (bit torrent).

Everyone who participates in the system agrees to trust this 1 party only, because they find them trustworthy

Even if the operator of the currency is perfectly benevolent and fair (ie, trustworthy), there is still a massive risk inherent to having a centralized point of control: what happens if the control falls into the wrong hands? (eg a rogue employee, hacker, government, etc). This is a massive risk, that only increases as the value secured by the network increases. What happens if the government policy changes? A war breaks out? The centralized entity running the network goes bankrupt? What if a natural disaster takes out their office? There are a lot of ways things could go wrong, and even if the chance of something happening is small, the impact could be massive.
On the other hand, the Bitcoin network has no hierarchy, no admin, no central point of control or authority. On the contrary, the network is governed from the bottom up, making it much more robust against many types of attacks.

Or they can just process transaction on a 1st come 1st serve basis and eliminate the need for blocks, just timestamp each transaction :)

This would basically amount to an .csv file with limited write access, which is rather uninteresting in light of the above.


What is a decent value for Time Lock Delta on a lightning network node? Looking at the settings of some nodes on 1ml.com, I noticed the Time Lock Delta variable could vary a lot. How does it affect a node, and what is considered to be a decent value for it?

The timelock value can be up to 14*144 = 2016 or the expected amounts of blocks to be mined with 14 days.

Generally the cltv delta is a trade of:

The higher you choose this value the longer your node can be offline without you rising to lose funds. However other nodes might select not to route payments through you as the routing process if hiring the chain can technically last as long as the cltv delta.

The reason why you will see various values in the gossip update_channel messages (which 1ml depicts) is probably implementations use different default values and have also changed the defaults with various versions. I think it was 144 for one of the implementations and it came down to 6 or 9. Eclair mobile wallet which mainly creates private channels that you don't see on 1ml users 2016 because mobile phones might have longer downtimes


PHP - Bitcoind sets up wrong transaction fee I'm using Bitcoin API  EasyBitcoin-PHP

And also i have my custom transaction fee, that i'd like to pay per each Kb. The problem is that when i used bitcoind v.0.10.2 that was fine and i saw my custom fee (0.0002) in each transaction.
But when i've updated to v. 0.13.0 - it just won't work as it should. Here is the simple code, that i'm testing now

$bitcoin = new Bitcoin($user, $password, $host, $port);

$result = $bitcoin->settxfee(0.0002); //returns true
$newTxId = $bitcoin->sendtoaddress('144YsrgtDmRSPC8wUBXLxByCcQAEbV1EUB', 0.001); //returns c3a2fceed1c25624aee8e957c644813bf38808260aad607e623c68e182371ac9

And as you can see in this transaction the fee is 0.000045 BTC, not 0.0002 BTC.

I know, that there was a bug in prev version of bitcoind according to fee rounding, but in new version this bug should be fixed.

Maybe there is another trouble here?

The value you specify for custom fee is in fee per kilobyte. Until recently, the "size" for fee computations was always rounded up to 1 kilobyte anyway, leading some people to believe that the amount they're setting is the absolute amount instead.

Your transaction is 225 bytes, and pays 0.000045 BTC fee. That is exactly 0.0002 BTC per 1000 bytes, so it is working as intended.


I accidentally transferred my Bitcoin Cash to Bitcoin (BTC) wallet in BitMEX! A few days ago, I bought Bitcoin Cash and transferred it to my BitMEX Bitcoin BTC Wallet by accident! This is money that I can't afford to lose! What do you suggest I do?

My conversations with BitMEX:

ME: Hi! Please help me

I just accidentally transferred my Bitcoin Cash to my BitMEX Account. I didn't realize that it needs to be Bitcoin and not Bitcoin Cash. Can you please reverse the transaction so I can fund my BitMEX account properly?

BitMEX: Unfortunately, our wallets are not capable of BCH withdrawals without extensive development and time, which means the funds are essentially unrecoverable/frozen at this stage. I have placed your details on a list of users who have deposited BCH, and we will contact you if/when we upgrade our wallets to enable withdrawals.

Currently there is no ETA on this.

ME: Are you telling me that the funds are unrecoverable/frozen at this stage, but once your wallets are upgraded, I would be able to withdraw? This means I have not lost my money, correct?

Other exchanges like BitPay have a feature that allows the option to "clone" the Bitcoin wallet address so it can be retrieved. How come BitMEX doesn't have this?

This is money that I can't afford to lose especially in short term. If lose it, I am going to end up losing everything and probably end up homeless :( :( :( :(

BitMEX: It may be recoverable at a later date, however, we cannot be certain of this.

Please note what is stated on the deposit page: `Do not send Litecoin, Bitcoin Cash, or Tether to this address! Unfortunately, these networks will accept Bitcoin multisig addresses as valid. Any unsupported coins sent here will be lost.

ME: I did some research and found that it is recoverable by BitMEX. BitMEX has the private keys to access my BTC Wallet on the exchange and your help desk should have a way to fix this. Why is BitMEX not trying to work on this now? Have you forwarded this to your technical team?

BitMEX: Our current wallet infrastructure does not support the new BCH chains and it will take us significant time and resources to access which we do not have right now. I cannot give you an update of time when we may be able to access this for you, but we are looking to try and solve this problem in the future. Please bear with us, this is the reason why we explicitly mention to NOT send us anything but BTC.

I'm going to not close this as off topic as OP has already contacted the customer support in question. Instead, I will try to answer why some exchanges/companies have such a stance.

Unlike traditional finance systems, crypto does not really have a refund system. If money is sent out of an address, there is no way to recover it without the receiver's cooperation. Due to this, crypto custodians (companies that hold crypto on behalf of others, such as exchanges holding their users' crypto) must take extensive steps to ensure that a rogue staff member, or someone who is able to gain access to internal systems, is unable to send the crypto to an address not controlled by the exchange.

A side effect of this pattern is that in the normal course of operation, private keys are often completely inaccessible to humans. In exceptional cases such as forks and disaster recovery, there will only be a handful of people who are able to access the raw keys/mnemonics and sign arbitrary transactions needed to recover full operations.

Moreover, such keys are often stored inside Hardware Security Modules, which impose further restrictions on access.

For an exchange such as bitmex, the time, effort, and security risk required to recover BCH from a BTC address for just a few users is likely not worth the engineering or human resources.

Many companies adopt such a stance to avoid compromising their security unnecessarily, and to safeguard funds for all users. Certain exchanges, such as Bittrex, have clearly run into this issue often enough that they have a system in place to recover some cross chain funds. However, that is the exception, not the norm.


what is the Difference between ./bitcoind and $ ./bitcoind -daemon? I was trying to interact with bitcoin network with JSON RPC commands, So I referred to the bitcoin wiki page https://en.bitcoin.it/wiki/API_reference_(JSON-RPC)#Java it shows to run the bitcoin using 'bitcoind -daemon' command. but another page of bitcoin wiki https://en.bitcoin.it/wiki/Running_Bitcoin suggest running the node using './bitcoind' command.so
 What is the difference between the two commands?

What is the difference between the two commands?

./bitcoind - runs bitcoin node in the foreground.

./bitcoind -daemon - runs bitcoin node as daemon.

  What is daemon?

In multitasking computer operating systems, a daemon is a computer program that runs as a background process, rather than being under the direct control of an interactive user.


When pool mining, how do I get paid? Just for fun i decided to check out bit coins and do some mining. No visions of making millions or anything I just find the concept pretty cool. At any rate I have installed guiminer in windows and i have installed the bit coin wallet from the bitcoin site.

How do i get my address so that i can have someone send me funds and in the mining software how do i set it up so that it knows where to deposit funds? I have it up and running using btc guild pool but it has never asked me about how to send me bit coins.

I am confused i feel  like i am missing the most important piece of the puzzle.

You need to get a receiving address from the client (or from an e-wallet if you prefer) and copy and paste it into the address field against your account in BTCGuild.

Bear in mind that if you're using the Bitcoin-QT client there is a very long synchronisation process to wait for, though you may be able to get a receiving address before it completes (but you won't see any received bitcoins in your wallet until it's synchronised). Depending on your hardware and bandwidth, synchronisation may take several days to complete.

While there's no receiving address against your BTCGuild account, you'll just be accumulating bitcoins against your BTCGuild balance, so you aren't losing anything in the meantime.

Bear in mind that the "difficulty" level of mining for the network as a whole has risen to a very high level due to the arrival of highly efficient dedicated ASIC bitcoin mining hardware. This means mining on graphics cards is likely no longer profitable (depending on purchase and electricity costs) and CPU mining hasn't been profitable for a long time.


Where can I find the raw stats data of the current bitcoin network? So that I can build something like this and this in PHP? Is there any official data feed or do I have to go through the entire network programatically to get them?

There's APIs like https://blockexplorer.com/q/ which provide the information, or you can just collect it yourself from your own bitcoind. Most of it is fairly straightforward, but there's a few which you need to do some analysis on if you generate it yourself (like difficulty estimation).


What are the differences in these methods of creating wallets offline? I have seen several ways of creating an offline wallet for cold (secure) storage:

Use BitcoinQT to generate a wallet offline using a clean install of Ubuntu.
Armory
Electrum

Are any of these methods more secure than the others? Is there a recommended solution for creating a secure offline wallet?

The problem with using an operating system like Windows to create these password-protected wallet is that you could still be offline (not connected to the internet), but if you have some sort of virus installed, they could copy your wallet, get your password that you'd type in, and store it locally. Then as soon as you connect it to the internet, they would send all this info to their servers and you're wallets would now be breached.

So using an operating system that is secure and open-source is very important. I would personally even disregard Ubuntu, because it has a graphical interface and it comes preinstalled with lots of questionable software - some of which has been recently set to send unwanted data to their servers.

Using something like FreeBSD or other Linux distros, directly using the command-line, and no graphical interface, would lower the barrier for any sort of attack.


Is there a safe way to use bitcoin based online services? Today I've seen new service www.spend-a-bit.com.
Idea is good, however you need to send them bitcoins first and then you get your balance.

Question is: Is there any way to make online services safe to use, without worrying that you send bitcoins, but never get anything back?

Examples, which come to my mind:
1) Some website takes responsibility as dispute system. You send them first, then receive balance, then they send it to merchant. In case of arguments they check if sides kept their promises. It always takes time to figure out, which makes it less attractive for scammers.
2) Just some public ratings for services. Where everyone can report scam

Look for merchants with a good history. There will, at some point, I imagine be a listing of good and respectable merchants, ranked based on consumer reviews. Until then, just use common sense. I have yet to be scammed with Bitcoin (or USD) because I rely heavily on the reviews of other trustworthy individuals before making purchases.
Don't send more than you can afford to lose. (Bitcoin is often still referred to as an "experimental" currency for a reason.)
Even an escrow service doesn't protect you 100%. In my opinion, you're better off relying on your own intuition when placing purchases, and not buying big ticket items with Bitcoin.
Use a reputable gift card service as an intermediary. I use gyft.com to buy amazon.com gift cards, which allows me to store value in Bitcoin and also get the consumer protection offered by Amazon. (This, of course, means that I trust gyft.com, which I do -- based on a history and other consumer reviews.)


Is the statement that LN hot wallets will be less secure than non LN hot wallets correct? I say this because of the following reasoning: Lightning network wallets will have to completely possess my private key, whereas non-LN hot wallets will most of the time only have my key encrypted (I sign the tx locally over my browser).

Of course this depends a lot on the implementation you are using but generally the answer is yes.

LN wallet vulnerabilites = hot wallet vulnerabilities + LN protocol vulnerabilites

Here are the main additional issues coming with running a LN node:

You could be scammed by somebody releasing an old channel state
You announce to the whole world how much money you have on your node [this is optional and depends on whether you want your node to be routing other peoples payments or not]


What if the miners allowed invalid transactions, and other nodes did validation? Here Peter Todd writes:

  [A]s an optimization
  Bitcoin goes a step further and disallows invalid transactions to be published
  in the blockchain at all, but that's the thing: that's just an optimization
  that full-nodes don't actually need to operate.

Maybe I'm misunderstanding this, but it seems to be an interesting idea: the blockchain can contain invalid transactions, but all validating nodes will ignore them. I'm thinking, what if bitcoin was designed this way, or if someone created such hard fork, could it work? If yes, what drawbacks should be expected?

Correct me if I'm wrong, but probably Counterparty already works in this way, since they use bitcoin blockchain.

I'm thinking, what if bitcoin was designed this way, or if someone created such hard fork, could it work? If yes, what drawbacks should be expected?

There are two I can think of:

No SPV

Right now, to trick an SPV client, you need to get 51% of miners to include an invalid transaction. However, if all miners include all transactions, then SPV is not possible.
No safe softforks

P2SH was a softfork that made it possible to send money to a hash of a script. If you didn't run P2SH enforcing code, enough miners ran it that your node would still agree with everyone else.

However, if rules are not miner enforced, then two clients running slightly different code can look at the same blockchain and get two entirely different pictures of who owns what.


Does MuSig have the same security as 2-2 multisig? Disclaimer: This question is of theoretical importance to me trying to educate myself better on cryptographic principles and signature schemes. I don't intend to imply that in practice schnorr signatures are less secure than current 2-2 multisig transactions / scripts.

I am currently reading the musig paper and about scriptless scripts. In my understanding an important common idea in both cases seems to be able to have a single signature produced from several private keys.

Let us assume I can brute force the private key from a public key within reasonable time let's say 1 month (for example because I have a somewhat efficient algorithm for the discrete log in ecdsa (which I don't have). Also assume I can invert the hash function of Bitcoin addresses quickly. Or assume we just know the public keys because I am the third party in an escrow service)

Wouldn't I be able to break a MuSig address within 1 month (under the above assumption) by breaking the aggregated private key to the aggregated public key whereas in the setting of a common 2-2 multisig wallet I would need 2 months in order to be able to provide two valid signatures since I'd have to bruteforce both private keys independently of each other?

Yes or no, depending on your definition.

You are right that the expected time to forge a 2-of-2 multisignature is twice that of a single signature, because you obviously need to use your forger algorithm twice.

However, in practice such constant factors are ignored when describing security levels. For example, typically ed25519 and secp256k1 are placed in the same group of 128-bit security, despite the fact that secp256k1 needs on average 4x more iterations of Pollard's rho algorithm to break the DLP. On the other hand, due to secp256k1's efficiently computable endomorphism, individual iterations of that algorithm are 1.7x faster than what would be expected otherwise.

Furthermore, the unit they're specified in is vague. When talking about ECDLP, the security levels is usually specified in terms of the number of elliptic curve multiplications. But an EC multiplication is not a trivial thing, nor is its performance identical across curves. However, when talking about things in the order of 2^128 a factor 10 here or there only changes the exponent by 3.3. It gets even fuzzier when you take into account specialized hardware that could be built for certain tasks, making it even hard to compare.

The point is that we don't care how long things take for an attacker. We only care that they're so long that no attacker could conceivably use them to pose a real threat.

If it takes too long to forge two signatures, it very likely also takes too long to break one.


For lightning network, can I surround myself with a bunch of nodes to increase anonymity? Let's say I'm on a Lightning Network. Can I create a bunch of nodes, and do all my spending through those nodes?
Can you hide the fact that these nodes are likely controlled by the same person?
How often would you need to contact the blockchain? Would doing so compromise privacy?
(You may assume that every network action is done from a different IP address.)

Every node you spin up will have to have Bitcoins that you can lock up to put into the payment channels it creates. Nothing is stopping you from starting a new node (and payment channel) for every node you transact with. And as long as none of the Bitcoins you used were connected to each other it wouldn't be very easy to prove those nodes were controlled by the same entity.


Is it possible to share blockchain data between multiple nodes? I'm planning to create a Raspberry Pi Cluster in order to run several bitcoin nodes. Can I share parts of the ~/.bitcoin/ folder in order to not have to store the entire hundreds of GB for each of the nodes? My idea would be that one of the nodes has an external hard drive with the bulk data and the others make use of it over nfs or smb.

Any ideas or even experience on this?

I don't think this is possible; you'd have multiple node instances trying to read/write a single data directory. This is not the intended operation, you'd likely end up with corrupt data.

If you must run multiple independent nodes and are worried about storage requirements, I'd recommend looking into 'pruning mode', it will limit the storage requirements for each node instance.

Note that you can copy the data directory folder of a synced node, so that you will not need to run through the initial block download and sync for each individual node (lots of savings in time and computation here). Just be sure to stop bitcoind before copying the data directory, and then start it up again once the copy is complete. This will help avoid potential data corruption/loss.


How does the reference bitcoin wallet/client handle rejected transactions? (in the API) I'm working on writing a wrapper for the standard bitcoin client. I'll be communicating with it via the JSON-RPC API and I'm just wondering if anyone has any experience with this;
this app is mostly geared to receive BTC, not send.

How does the client react when it sees an incoming transaction from an address of x BTC, but it is later determined by the network that that transaction was invalid (probably due to a double-spend by someone sending BTC to an address, then immediately sending the same BTC to another address but from a far away play so the lag allows both to go through. In this case I'm talking about transactions before a block is mined)

Does the transaction just disappear off the network and away from the bitcoind interface/API or is that recorded somewhere? (In the confirms result in the API?)

I want my app to be able to keep track of any instances of this happening and notify the end user.

Thanks.

With Bitcoin-Qt/bitcoind there there is no callback or other real-time notification that a transaction that had previously been a valid 0/unconfirmed is not a transaction that will not confirm due to there being another confirmed transaction for the same unspent transaction output (UTXO).

When a new block is found, each transaction can be inspected (using Raw transactions) to determine the INPUT(s).   If the input was already used by a previously seen transaction, then the spend from the block would cause the earlier transaction to be an invalid double spend and it can be purged.

The Bitcoin-Qt/bitcoind client will no longer display the 0/unconfirmed transaction for a payment received that becomes invalid after a double spend from a new block arrives.


Why wait for several confirmations before shipping? Suppose customers pay me bitcoins for their purchases. As far as I understand, I could check block chain that the proper payment amount has been made to the address I generate for each customer, and after waiting for several confirmations I should then ship the product. Why several confirmations? Wouldn't one be sufficient?

Also, is the method I described above the best way to identify a paying customer?

Thanks

After one confirmation, the transaction is very unlikely to be double spent. However, each further confirmation reduces the probability exponentially. Basically, the number of confirmations provides additional security that the transaction will not be reversed.

One reason a retail store or individual might wait for more confirmations to ship is time; it only takes an hour for 6 confirmations to go through, which is negligible time for shipping a product, but can provide assurance to the seller.


At what point does a hard fork occur? with the whole debate and concern over a looming Bitcoin hard fork I would like to ask at what point has the hard fork occurred? Is there a specific moment when developers decide to implement BTC/BTU code that miners and nodes then decide to use or is it more gradual where miners and nodes slowly start leaning more towards one or the other?

If it is the case of pushing a the button and uploading/implementing new code will there be a publicized date?

A hard fork is something that happens when some of the nodes on the network follow a new set of rules that are in some way incompatible with the existing set of rules.  It occurs upon mining the first block that is valid under the new rules, but invalid under the old.

The software following new rules is deployed and in place before the fork occurs, but doesn't actually start using those new rules until some triggering event.  In practice, this event is when a certain percentage of the last X blocks have been mined by nodes supporting the new rules.  This event is called activation of the new rules.

The fork, however, doesn't occur until a block is found by the nodes following the new rules, that is invalid to blocks following the old:

[G] [G']
 |   |
[F] [F']
 |   |
[E] [E'] <-- Hard fork occurs when this block is mined
 | /
[D]
 |
[C]
 |
[B] <-- "forking" nodes start following new rules here
 |
[A]

Here we see an example where the nodes following the new rules activate at block B.  However blocks C and D are valid under the new rules, and the old rules, thus they don't cause the fork.  When block E' is mined, it is only valid under the new rules, meaning that it is rejected by clients who follow the old rules.  Those clients that reject the block go on to mine E, and we now have two irreconcilable chains.


"Shared wallets will not work correctly." One satoshi bones rule states:

SHARED WALLETS WILL NOT WORK CORRECTLY.

What is a shared wallet? Why would shared wallets not work correctly with satoshi bones?

A shared wallet is often what online services and exchanges often use. They aren't fully functional wallets, and rely on a backend service to manage funds across users, this is the "sharing".

The reason you can't send from a shared address is because shared wallets only allow 1-way transactions due to technical limits. Satoshi Bones sends back bitcoins to the address it receives from after betting from what I'm assuming.

You're using a shared wallet if your sending from an exchange, Coinbase, and most online wallets or websites. The only web wallet that is not a shared wallet that I know is blockchain.info. Else just use wallets you install like Bitcoin-qt, Armory, or Electrum. I recommend Electrum.


Why do you need the *complete* blockchain to mine bitcoins, why not just the last n blocks? I understand that to mine bitcoins you need the hash of the previous block, and in order to resolve possible forks. But why not just have the last 100 blocks (or more whatever is appropriate) and keep a sliding window of the last blocks?

To mine a valid block, you need to ensure that every Bitcoin transaction in your block is valid.  Because each transaction (except for coinbase transactions) spends the output from a previous transaction, you need to have processed every previous transaction in the blockchain.

This has a couple implications:

You need to download and process the whole blockchain before you can start mining (provided you only want to produce valid blocks).
You can create a database of just the information you need to validate future transactions and only store that, allowing you to get rid of the rest of the blockchain data you downloaded and processed.  Bitcoin Core 0.11.0 began supporting this blockchain data pruning feature for mining with the -prune parameter.
You can outsource validation of transactions to someone else and only mine the transactions they say are valid.  This way you don't need to store any blockchain data at all.  This is what most pool miners do.


P2SH that pays out only to a certain Bitcoin address? Is it possible to create a type of redemption script that when the correct number of signatures are collected pays out to a pre-defined address ?

I believe this is possible, but with a subtle difference to how OP suggests.

OP seems to want: funds are first signed off by the multisig address, then a transaction is broadcast moving the funds to [destination].

It's not possible for a destination script to stipulate where the funds can end up. A new opcode could introduce that feature, but we can do this a different way.

I would argue this no weaker than having a transaction between the two destinations (as OP envisioned). In either case the multisig participants could hold the funds to ransom before the end user can move them.

Multisig: 2 [pubkey1] [pubkey2] 3 CHECKMULTISIG
Destination: DUP HASH160 [hash] EQUALVERIFY CHECKSIG

Since we cannot lock funds so they are released to a certain destination, we just merge the two scripts, allowing predefined party to take control only when the multisig is solved. He would propose a spend to the others, they sign, then he signs (it can be done in any order)

The scriptPubKey (or P2SH script) would then look like this:

 2 [pubkey1] [pubkey2] 3 CHECKSIG DUP HASH160 [hash] EQUALVERIFY CHECKSIG

And the scriptSig:

 [sig1] [sig2] [sigA] [pubkeyA]


Is P2SH-wrapped P2PKH possible? There is a known form of address that called P2SH-wrapped SegWit, wrapping SegWit address as Base58 (P2SH) address.
Thinking about such address type, I come up with the question: Is P2SH-wrapped P2PKH possible?
I know that it is completely useless and meaningless, but I want to think about it now.

Step 1: Decide an address to convert

Let's use the address 1CciesT23BNionJeXrbxmjc7ywfiyM4oLW as it is used as a sample here.

Step 2: Convert it into "output script", then hash it using "hash160"

I got OP_DUP OP_HASH160 7f6ab65fa911f558ca2dde3e9d073acb02c0d5c6 OP_EQUALVERIFY OP_CHECKSIG.
Hashed result is c852ac34a1c76b63a279c97502c9ccc4e3cb9e8b.

Step 3: Make P2SH address from hashed script

toBase58Check(c852ac34a1c76b63a279c97502c9ccc4e3cb9e8b, 5) = 3KxE77EHe1ip6WGRifwr9fZ5WBDGsLyWFz
The result 3KxE77EHe1ip6WGRifwr9fZ5WBDGsLyWFz is what I want.

Here is the code to reproduce this address generation.

Question

Is that possible to spend coins sent to 3KxE77EHe1ip6WGRifwr9fZ5WBDGsLyWFz (P2SH-wrapped P2PKH address), like how we spend coins sent to P2PKH addresses?

It's definitely possible and you've done the nesting correctly.
Also, aside from being "useless and meaningless", it's more expensive since the scriptSig for redeeming outputs sent to this address will have to contain the 26 bytes of the redeemScript (the p2pkh script itself) and the push operation of it, along with the public key and signature.


Is there a way to find how many nodes are there? As a single bitcoind node I can only detect how many connected peers I have. Is there a way to have a vision of the whole bitcoin network?

There are websites such as Bitnodes.io that track the number of nodes currently reachable in the network. The site also gives an account of the versions the nodes deploy and what countries they are located in.

According to Bitnodes.io, there are currently 6426 nodes reachable.

To collect your own list of nodes, you'd ask your peers for more peers, i.e. getaddr, then recursively request more peers from the newly discovered peers, until you have polled everyone you have discovered.

Note that this doesn't guarantee you to find all nodes, as some don't accept connections from the outside, and getaddr doesn't necessarily give you all peers. Also see: Does getaddr.bitnodes.io find all Bitcoin nodes or only one node per mining pool?


Does having a pruned node reduce its network score? My node's network score (revealed by getnetworkinfo) is very low (<75). Most of the time there are no peers with incoming connections connected to the node. Is this because I'm running a pruned node? Would that reduce the network score?

What's the meaning of the score in getnetworkinfo?

When you look up getnetworkinfo in the Bitcoin Developer Reference, it says about score:

  The number of incoming connections during the uptime of this node that have used this address in their version message

If I understand that correctly, this means that the score is the number of times incoming connections have been established with your node since it started.

Network services and pruning

Nodes on the network announce the services they offer by setting their network services flags.

There are currently five flags in use:

NODE_NETWORK (1)
NODE_GETUTXOS (2)
NODE_BLOOM (4)
NODE_WITNESS (8)
NODE_XTHIN (16)

Out of these, NODE_NETWORK indicates that your node will serve historical blocks. Currently there is no way to announce that you can only serve a subset of all blocks, so just like nodes that cannot serve blocks at all, pruning nodes don't advertise NODE_NETWORK.

Pruning nodes don't get referred by DNS seeders

This has a direct effect on the number of incoming connections pruning nodes get, as DNS seeders only refer nodes that advertise NODE_NETWORK when new nodes request additional peers to connect to. This means that pruning nodes only get incoming connections when they are referred by non-DNS seeder nodes.


Do transactions get slower over time? The number of blocks formed is 1 every 10 minutes. As the number of transactions increase, wouldn't the time taken for a transaction to confirm become slower?

I am asking this because this the number of people transacting increases but how fast blocks get generated is constant. Or am I going about it all wrong?

From what I read, I understand that a block is generated whenever a proof of work is submitted by a miner; this happens when the miner processes a transaction. So in addition to the transaction, he also gets some bitcoins which are once again added to the block chain.

I understand that a block is generated whenever a proof of work is
  submitted by a miner; this happens when the miner processes a
  transaction

That's not correct. The miners doesn't submit a found block (proof-of-work) when he processes a transaction. He does when he found a block hash that was valid.

That can take a long time and in the time he looks for a good block, he will receive probably thousands of transactions.

He will verify those as they come in and put them in the block he is currently working on, all of them, one by one (until the limit is reached). When he then finds a valid block hash, he will submit the entire block consisting of all, or most of, the transactions he received.

So, to answer your questions, transactions do not necessarily get slower over time, on the condition that the upper limit of transactions in a block is high enough. If the limit is too low, there can arise a bottleneck for transactions trying to be verified.

FYI: You can see that all miners have all those transactions in their working block, but only one of them can submit the block. To prevent transactions from occurring twice, once a miner receives a new valid block from a fellow miner, he checks what transactions in his pool are also in that block. He will remove them from his pool and continues mining for the next block with the transactions from his pool that were not in the block of the fellow miner.


The halvings occur in form of a bitshift operation. So, in other words, you halve and cut off. Therefore, the rewards should be rounded down.

A while back, I've generated a table with block rewards and other information.

The rewards in satoshi per block should work out to be:

5000000000
2500000000
1250000000
625000000
312500000
156250000
78125000
39062500
19531250
9765625
4882812   <--- first deviation from your table
2441406
1220703
610351
305175
152587
76293
38146
19073
9536
4768
2384
1192
596
298
149
74
37
18
9
4
2
1
0


Multibit password incorrect - what else can cause this? I have my balance cash display on my Multibit account at left top of my account which was my last transaction. But to send it out to another bitcoin account, it keeps telling me that "The wallet password is incorrect." I am fully sure, I have never changed my password and I have tried the directory given but I couldn't get it done, even none of my old transactions are showing up again. Do password regenerate itself to give new one or is my account file corrupt?

Because I know that I run some program into my computer recently i.e. photoshop program, could
that be the cause? But I never tampered with bitcoin file - that is what I just can't figure out
here. Please help. I am using MultiBit version 0.5.18 in Windows 7 Ultimate please help. Or
is there anything else for me to try that I have not done?

It could be a problem with the Multibit app.

Did you export before the private keys ?

If yes, You may import your keys at blockchain.info (just paste the .key data to import form)


What is the input and output script of the funding transaction? I'm looking for what the state block look like on the blockchain to set a channel on the lightning network.
I found everywhere that the input script is a multisig script (since the two people that set the channel will put there state on the blockchain), but I can't find how they lock the amount until the channel is closed.

So I have two questions :
What is the script of the output script of the state block (the block that starts the channel)?
Where to find such information ?
If you have the scripts of the block that is published on the blockchain after the channel is closed, I would gladly take them also :)

Thank you for your help!

What is the script of the output script of the state block (the block that starts the channel)?

The output script of the funding transaction is a P2WSH output where the witness script is a 2-of-2 multisig. Because it is P2WSH, you cannot see this script.

This locks in the amount because the output with that script has a specific amount (the amount the channel is being funded with) and the only way to spend that output is with both parties agreeing to spend it.

  Where to find such information ?

All of this information is specified in the Lightning BOLTs, specifically BOLT 3.


git checkout v0.16.3 gives error Trying to update my bitcoin core installation on raspberry pi with git checkout and it's giving the error "error: pathspec 'v0.16.3' did not match any file(s) known to git." Any recommendations?

That error likely means that you have not done a git pull or git fetch. Those are required so that you can pull down the latest source code and tags so you can then do git checkout.


What happens when transferring bitcoins to no address? What happens on the blockchain if I try to transfer bitcoins without providing  a recipient address?

Most wallet software won't let you send to a null address. If you have a transaction without an output script, that's invalid and won't be accepted by the network. If you have an output script that's simply an OP_RETURN or otherwise unspendable, those coins are lost forever.

Generally, you should be using a wallet to transfer bitcoins and if you don't provide an address using a wallet, you'll most likely not be able to send it.


Help! Tried sending bitcoins to myself and they disappeared I'm using the bitcoin-qt client on v0.8.1-beta on Mac OSX

I have 3 addresses.  I tried sending .01BTC from one address to another.

1K3BjT1y2X2v3PGhqHh33dRb6kVGdR5CBA ->  1MattHdtuFSof3ctrZhTGzsfQecQBso2nw

However!!! When I lookup the transaction ID: f638b5ecda8d8b0280a49f342e6cac13560917d5e678f1e3b9a7f35f69b15b97

I did sent the .01 to myself but it also sent
1.2594786 BTC -> 1Ypzoqn4BYSbgeqPJ3mSPdqBC8JGzgX4r

I have no idea what the 1Yp... address is!!

What happened!!!  Did I just loose $1000 ?

it did NOT prompt me about paying any transaction fees!!

I would greatly appreciate any help!

Update:
Yeah!!! I recovered my coins.   For whatever reason bitcoin-qt generated a new private key but didn't show ANY indication of this in the GUI.  This may be because it was backlogged by about 100 days in downloading transactions..  not sure.  but in any event LUCKILY i had been messing around with trying to migrate to multibit.. anyway i tried to export the private key and was successful:

walletpassphrase  120
dumporivkey 1Ypzoqn4BYSbgeqPJ3mSPdqBC8JGzgX4r

Then it worked !  it actually dumped the priv key and I was able to import into blockchain.info and get access again.  Not sure what happened but it was very unnerving...   again, the GUI does NOT show 1Ypzoqn4BYSbgeqPJ3mSPdqBC8JGzgX4r as one of my addresses...

Anyone care to explain what happened?

Source:

What you are seeing is a "change address".  When you send an amount that is less than you originally received, it sends the whole amount, and returns the difference to your wallet as "change".  This change may not be spendable until the transaction is confirmed.  The change address is different each time, which makes it harder for a third party to tell for sure how much is sent to which wallet.  If you are not interested in privacy, you can use a different wallet (such as Electrum) which has an option to not use change addresses -- in that case the change would go back to your original address instead of to a new change address.


How to remove a wallet from Android Bitcoin Wallet I'm using Bitcoin Wallet for Android.  I recently backed up my wallet's keys and did a fresh install to a new device and restored my wallet there.  All great so far..

I'd now like to remove the fresh empty wallet that the app created for me when I did the install, but can't find a way to remove it from the application's address book.  Anyone know how I might do this?

Okay, so I think I misunderstood what this app is showing me.  A bitcoin wallet can (and often does) have multiple addresses (public+private keypairs).  What I was seeing in the app was not multiple wallets, but just multiple addresses.


How to start multiple altcoin rpc daemons at once? I'm building an ecommerce website where I want to do my own payment processing in seven different ctyptocurrencies. I want use the separate versions of bitcoind for each currency, because I feel like that's the most straightforward way to do it (unless I'm wrong). I've even come up with some dog simple interface code to use once I get them all running!

So I've compiled each one, and I've got my rpc config files ready and raring to go, but whenever I start one of the rpc servers... it doesn't exit to the command line. It either just sits there, or depending on whether or not I'm printing to the console, it tells me everything that's happening.

When I hit cntrl+c, it exits to the command line, without stopping the daemon, which is what I want, but I can't seem to figure out how to write a script that will start all of the different rpc servers at once.

I have to be missing something stupidly basic.

I've looked everywhere on this stack exchange, and have not been able to find a solution to this problem.

HELP!

Thanks in advance for your time and attention.

Edit: I'm attempting to do this from the Linux command line with the standard compile parameters.

in seven different ctyptocurrencies

Not a good idea. Trust me. Bitcoin is enough, and for that you don't need any daemons. Bitpay or blockchain.info can handle it for you.

Well, here is what I do to start Litecoin daemon:

  "C:\Program Files (x86)\Litecoin\litecoin-qt.exe" -server -rpcallowip=127.0.0.1 -rpcuser=user -rpcpassword=password -rpcport=1488


tx-malleability who re-signed the tx? It would be really great if there was a way to loop through all the txids in a new block and be able to check if you (can) sign your original transaction the same way it was re-signed - that it matches the new txid

but i guess it would require that you also hold the private key of whom ever resigned your tx :/

Question(s):

if I somehow had the public key of whom ever re-signed my tx what could i observe from the new malleat-ed txid?

would i be able to run a test on the new txid to see if it is somehow related to my tx and/or my original txid?

If so how would I go about accessing the public key that relates to the private key that re-signed the txid in a new block?

Assumption(s):

the public key is named public because i could possibly/somehow have access to it easily...(??)

If the pub key is not readily accessible (massive assumption) peers on the bitcoin network should supply the public key part of their keys (to other peers) that is the pair to the private key they use to malform/re-sign transactions...(!!?)

Actually, nobody else can create a valid signature for your transaction inputs, as they do not possess the private keys to do so.

When a transaction is malleated by a third party, it is not re-signed, merely it's signature is changed. ECDSA signatures are malleable, in that taking the negative of number S (modulo the curve order) does not invalidate them. As the TXID is created by hashing the complete transaction data, this will result in a new valid transaction with identical payload except for a new TXID.


How can I check if my seed matches a BTC address? I have a seed with 12 words from Coinbase Wallet. The derivation path is: Segwit Native (p2wpkh) m/84'/0'/0' BIP84.
The first address generated seems to be wrong because it does not get generated anymore.
I can't get access to the funds with Electrum and a gap limit of 10000.
I'm sure that the address contains my funds and no other transactions than my initial are visible in the blockchain explorer.
How do I do get access to that address?
Can this be somehow brueteforced or something?

In HD wallets it rarely happen that a gap of more than 20 is being used. It relates to bip32 standard in most cases. And it may happen in only few cases:
-if the receiver is a payment processor that puts out hundreds of invoices and only few are being paid.
It doesn't sound like your case though. I would suggest using IanColeman website, please remember to use it in offline mode and be very careful about it anyway. I wouldn't use it with a bigger amount of funds personally.


bitcoind not booting, tons of output in debug.log file running $ bitcoind -server just hangs, when I look at my debug.log file it's going really crazy:

2016-03-19 01:26:51 UpdateTip: new best=00000000000005f9ff6fe1e9e19f6f64dd9c1167077b7de4b93146df2e6f1004  height=192099  log2_work=68.48247  tx=5576861  date=2012-08-03 09:07:41 progress=0.022371  cache=46.6MiB(128445tx)

The docs show me that running bitcoind should have asked me to set a password, but I never got that. Is something going wrong? Thanks

Turns out that $ bitcoind is just downloading the blockchain, the docs here: https://bitcoin.org/en/full-node seem to be incorrect. :\


Raspberry Pi cpuminer error: -bash: ./minerd: No such file or directory I created a cpuminer instance for my Raspberry Pi derived from this tutorial.  Apparently, I've correctly configured the implementation because I can run the example provided in the README:

./cpuminer -u alexanderjsingleton.Pericles -p pass -o stratum+tcp://stratum.slushpool.com:3333 -t cputhreads

Apparently, that command is indeed the example because I'm not seeing any feedback displayed within my slushpool-account dashboard for that worker.
 However, according to this StackExchange:Bitcoin post, the appropriate cpuminer-syntax should be:

./minerd -u alexanderjsingleton.Pericles -p pass -o stratum+tcp://stratum.slushpool.com:3333 -t cputhreads

but that command returns the following error:

   ./minerd -h --algo scrypt --scantime 33  --url stratum+tcp://stratum.slushpool.com:3333 --userpass [alexanderjsingleton.Pericles]:[password] --threads 1 --retries -1 -P
-bash: ./minerd: No such file or directory

In any case, this is my first experiment with actual Bitcoin-mining, so I'd appreciate any guidance whatsoever to start mining on my first block- thank you!

The original CPUMiner software made an executable named minerd. You are using a fork of the CPUMiner software which changed the executable produced to be named cpuminer. The minerd ... command will not work because that executable file does not exist as it is not created. If you are looking up any instructions for using CPUMiner and it says to do ./minerd ..., just do ./cpuminer ... instead. Note that because you are not using the original CPUMiner software not all example commands and options will be the same or result in the same behavior.

There are multiple reasons that you do not see anything on your Slush dashboard. First of all, you are connected to a Bitcoin mining pool and yet you are using the Scrypt PoW algorithm. Bitcoin does not use Scrypt, so whatever you are mining is invalid. Secondly, your hashrate is so low that you are not finding any shares (besides the fact that they will be invalid as you are using the wrong algo), so Slushpool does not know anything about this your worker as it has not received anything back from it.


Safely buying bitcoins in person What is the safest way to exchange bitcoin for cash in person.

Obviously meet somewhere public but other then that how should an exchange proceed? What steps should be taken to ensure both parties are protected and everything goes smooth.

This is the usual procedure:

Meet in a public place with wifi
Person buying bitcoins gives you the cash.
Person selling bitcoins counts the cash.
Person selling bitcoins transfers bitcoins to person buying bitcoins.
Both people wait for 1 confirmation.

If you use localbitcoins, you can skip step 5.


What is it that makes BIP149 safer to activate than BIP148? There are currently two BIPs floating to enable a "user activated softfork" of Segregated Witness.

BIP148 is meant to activate by flag day on August 1st 2017 whereas enforcing nodes stop accepting blocks that don't signal readiness for segwit.

BIP149 proposes to initiate a second activation process based on BIP8 once the current versionbit (BIP9) activation times out unsuccessfully. From my understanding, BIP8 is the same as BIP9 except that at the end of the activation cycle it goes into LOCKED_IN instead of timing out.

Isn't it just a semantic difference, whether a subset of all nodes defines the proposal as LOCKED_IN or the same subset of nodes outright rejects non-signalling blocks until the proposal goes to LOCKED_IN?

What makes BIP149 safer to activate?

I don't see a significant difference or that the former would decrease the chance of a chainsplit significantly except for the longer waiting time until it is activated.

BIP 149 is different in that it does not enforce orphaning of blocks that don't signal readiness for SegWit. Instead just the supporters enforce the rules on any blocks that contain SegWit data and allow regular blocks to be mixed into the chain. Thus, miners that don't change anything will build on the SegWit chain.

To split the network, miners would need to deliberate fork the chain, either by continuously rejecting every SegWit block and outpacing the BIP149 chain, or by mining a block with an invalid SegWit transaction that legacy nodes accept and SegWit nodes reject and building on top of that.


Bitcoinj: Multiple wallets In my Java application I am now required to handle multiple bitcoin wallets (HD wallets?). This means sending and receiving BTC. Unfortunately bitcoin is quite new for me, so there are some questions:

Would it be possible to use BitcoinJ in order to do this? Is it performing well with hundrets of wallets (or more)? If not, what would be the alternative?

Regarding the workflow:
Of course the private key will stay on the user side and thus cannot be read by the application itself. Would it still be possible to raise events like "money received" without having the private key of the wallet?

I had the same doubt. I've done a little research in the library bitcoinj, and I think that it is possible to use multiple wallets at the same time. I recommend you first work with a single wallet so that you can understand a little how it works the library. I leave an example:

public class MainClass {
    private static final File BLOCKCHAIN_FILE = new File("block.dat");
    private static final NetworkParameters NET_PARAMS = MainNetParams.get();

    public static void main(String[] args) throws Exception {

        List<Wallet> wallets = getWallets();
        BlockStore blockStore = new SPVBlockStore(NET_PARAMS, BLOCKCHAIN_FILE);
        BlockChain blockChain = new BlockChain(NET_PARAMS, wallets, blockStore);
        PeerGroup peerGroup = new PeerGroup(NET_PARAMS, blockChain);

        for (Wallet w : wallets) {
            peerGroup.addWallet(w);
        }

        //Starting peerGroup;
        peerGroup.startAsync();

        //Start download blockchain
        peerGroup.downloadBlockChain();
    }

    public static List<Wallet> getWallets() throws UnreadableWalletException {

        List<Wallet> wallets = new ArrayList<>();
        for (int i = 0; i < 5; i++) {
            Wallet w = Wallet.loadFromFile(new File("wallet_" + i + ".dat"), null);
            wallets.add(w);
        }

        return wallets;
    }
}


OSX Bitcoin Core frozen on login while "Activating Best Chain" I recently downloaded Bitcoin Core for OSX, sent a payment to my wallet address, then waited for the client to complete a sync. About 2 days later and w/ only 8 weeks to go, the client froze. Upon restart, it only gets to "Activating Best Chain" and won't open fully.

I've seen related answers to this question in the forum, but I think they're all for Windows machines.

I do have my wallet encrypted.

I'd like to export my wallet to another client, but apparently am not using the correct inputs in console. Any help much appreciated.

cheers

...and when I try to run:
getwalletpassword "pwd" 600
the application crashes and I have to restart. So it doesn't look like the standard way to get my privkey out will work.

I've just fixed the same problem by deleting a circular symlink:

lrwxr-xr-x    1 2can  staff       17 Mar  7  2015 bootstrap.dat.old -> bootstrap.dat.old

in my data folder. The db.log file contained only one repeating error:

Too many levels of symbolic links


Authenticating with npm nodejs bitcoin-core module https://github.com/seegno/bitcoin-core
After using npm to install, I get an auth error when I try to run it.
The github repo isn't clear how to auth...

const Client = require('bitcoin-core');
const client = new Client({ network: 'mainnet' });

client.getInfo().then((help) => console.log(help));

I was able to auth using the older package: https://github.com/freewil/node-bitcoin

Using but with my own credentials:

var client = new bitcoin.Client({
  host: 'localhost',
  port: 8332,
  user: 'username',
  pass: 'password',
  timeout: 30000
});

Really lost how to do it on the newer one.

I tried the following, and it works:

const Client = require('bitcoin-core');

const client = new Client(
    {
        username: 'yourUsernameHere',
        password: 'yourPasswordHere',
        port: '8332'
    }
);

client.getInfo().then((help) => console.log(help));


Why the txn_count is zero in the block message? Why I'm receiving always zero in the txn_count field of the block message after calling it with a getdata ? Is this expected ? How can I get the transactions present in a block ?

I solved the problem, I was reading the Nonce from the block as a 64 bit integer like in the version message instead of a 32 bit integer.


Why was Armory and Blockchain wallet removed from bitcoin.org? I would like to learn from other's mistakes and avoid problems in the future. The page which shows available wallets is https://bitcoin.org/en/choose-your-wallet

Blockchain.info has suffered from numerous security vulnerabilities, some of which are discussed here:

https://github.com/bitcoin-dot-org/bitcoin.org/pull/663

Wallet listing criteria can be founds here:

https://github.com/bitcoin-dot-org/bitcoin.org#wallets

The reasons for Armory removal are explained here:

"Armory is no longer developed by ATI. Its website, bitcoinarmory.com, is down and is no longer Armory's website. The source code is now maintained by goatpig."

https://github.com/bitcoin-dot-org/bitcoin.org/pull/1235

A new pull request to bring it back is now open:

https://github.com/bitcoin-dot-org/bitcoin.org/pull/1301

the resolution timetable remains unclear:

"goatpig commented 5 days ago

@ABISprotocol I don't control bitcoinarmory.com, that belongs to ATI. Their current policy (I believe) with that domain is to offer binaries for the code they fully own, i.e. 0.93.x versions of Armory.

You could say the current state of affairs is that ATI tolerates my fork (and me using their trademark), but have yet to make a definitive move in either embracing it or opposing it (in which case I'd have to rebrand)."


Sync too low when using BitCoinJ to create and host own wallet I need to use BitcoinJ to create and host wallet on my own website. However, it keeps downloading blocks and the sync seems taking forever. I need to be able to create a wallet within a few seconds.

Could anyone please advice what should i do to optimize the sync time?

ECKey key = new ECKey();
Address addressFromKey = key.toAddress(params);

Wallet wallet = new Wallet(params);
wallet.importKey(key);

File blockFile = new File(“/tmp/bitcoinblocks”);
BlockStore blockStore = new MemoryBlockStore(params);
BlockChain chain = new BlockChain(params,blockStore);
PeerGroup peerGroup = new PeerGroup(params, chain);
peerGroup.addPeerDiscovery(new DnsDiscovery(params));
peerGroup.addWallet(wallet);
peerGroup.start();
peerGroup.downloadedBlockChain();

To make the sync faster and the wallet creation you can do by setting the checkpoints and later at the time of calling the create wallet method you can set the checkpoints for faster sync. Like I am using walletKit method of bitcoinj for wallet creation find the below code for example

WalletAppKit kit = new WalletAppKit(params, new File(walletPath), filePrefix);
kit.setCheckpoints( ReadProperties.class.getClassLoader().getResourceAsStream("checkpoints-bitcoin"));
kit.startAsync();
kit.awaitRunning();


Transaction net gain/loss I have a TXid in my wallet that contains multiple transactions (multiple sends and receives). From what I read these are transactions the network creates facilitate splitting/combining multiple inputs/outputs etc. When I gettransaction in the wallet I have I see

gettransaction 3dda59bf9801f88...5498dd7fd1c3f9577963b94
{
  "amount": -17.00000000,
  "fee": 20.00000000,

  "details": [
    {
      "account": "",
      "category": "send",
      "amount": 0.00000000,
      "vout": 0,
      "fee": 20.00000000
    },
    {
      "account": "",
      "address": "GPP7ic67...9TgitCB21r",
      "category": "send",
      "amount": -20.00000000,
      "vout": 1,
      "fee": 20.00000000
    },
    {
      "account": "",
      "address": "GPP7ic67...9TgitCB21r",
      "category": "send",
      "amount": -17.00000000,
      "vout": 2,
      "fee": 20.00000000
    },
    {
      "account": "TestWallet",
      "address": "GPP7ic67...9TgitCB21r",
      "category": "receive",
      "amount": 20.00000000,
      "vout": 1
    }
  ],

So, I'm trying to figure out what the net result of this transaction is. I received -17 and I had 20 fee. Does that mean I net loss 37?

hmmm, yes, this accounting system within bitcoin :-) bitcoin.SE is full of questions according to the balance on accounts.

The "-17" would indicate, that funds left from a known address in your wallet to an "external" target address. And if fees is 20, then yes, you have "outgoing" 37 coins. Btw: what system are you playing on? Addresses starting with "GPP"...?

When I was playing with the regtest env, I discovered that only "gettxout" provides reliable info, which tx might have funds. See also the thread here.

Here a short linux shell script, to fetch the idea. It needs to get updated to cope with many inputs, so one would need to iterate through the VINs...

  b_cli="bitcoind -regtest"
  FUNDING_TXID=3dda59bf9801f88...5498dd7fd1c3f9577963b94

  TX_PREV_VOUT=$( $b_cli gettransaction $FUNDING_TXID | \
                  awk '/vout/ {  print $2; exit }'    | tr -d "," )

  $b_cli gettxout $FUNDING_TXID $TX_PREV_VOUT

  # as per Pieter's comment, spendable VINs return "something"...
  if [ ${#ret_val} -gt 0 ]; then
    echo " ## This tx has spendable funds"
  else
    echo " ## no spendable funds on $FUNDING_TXID"
  fi

Adding up all the parts, which have spendable funds, would then be "your" balance. And you could check "in" and "out" going funds in your wallet.


How to hash data from 'getblocktemplate' request I've been reading lots of articles, trying to understand how to hash the data returned from getwork; but I totally overlooked getblocktemplate.

I'm just trying to understand how all the data below is put together to generate hashes, in order to form a valid block (using various nonces).

Apologies if this has been answered already. I know there are lots of articles on how to put together a block header, and using midstate, etc. Most of that I understand; but the data composition of getblocktemplate is quite a bit different, and really threw me off.

If someone would be kind enough to explain the whole hashing process (from a mining point of view), I'd really appreciate it. A programmatic explanation would be ideal, as I'm trying to make my own mining client.

{
    "result":{
        "version":2,
        "previousblockhash":"00000000000000075f2f454573766ffae69fe41d6c7ccfcabbf8588fcd80ed52",
        "transactions":[
                  {
                        "data":"0100000001cba672d0bfdbcc441d171ef0723a191bf050932c6f8adc8a05b0cac2d1eb022f010000006c493046022100a23472410d8fd7eabf5c739bdbee5b6151ff31e10d5cb2b52abeebd5e9c06977022100c2cdde5c632eaaa1029dff2640158aaf9aab73fa021ed4a48b52b33ba416351801210212ee0e9c79a72d88db7af3fed18ae2b7ca48eaed995d9293ae0f94967a70cdf6ffffffff02905f0100000000001976a91482db4e03886ee1225fefaac3ee4f6738eb50df9188ac00f8a093000000001976a914c94f5142dd7e35f5645735788d0fe1343baf146288ac00000000",
                        "hash":"7c90a5087ac4d5b9361d47655812c89b4ad0dee6ecd5e08814d00ce7385aa317",
                        "depends":[],
                        "fee":10000,
                        "sigops":2
                  },
                  ...
            ],
        "coinbaseaux":{
            "flags":"062f503253482f"
        },
        "coinbasevalue":2501100000,
        "target":"0000000000000026222200000000000000000000000000000000000000000000",
        "mintime":1379549850,
        "mutable":[
            "time",
            "transactions",
            "prevblock"
        ],
        "noncerange":"00000000ffffffff",
        "sigoplimit":20000,
        "sizelimit":1000000,
        "curtime":1379553872,
        "bits":"19262222",
        "height":258736
    },
    "error":null,
    "id":"curltest"
}

The wiki has an extremely good explanation, with a python run-through that takes you from GBT output to share submission. You could also have a look at some of the implementation in Luke-Jr's "gmp-proxy", which translates GBT reponses into a getwork compatible interface.

https://en.bitcoin.it/wiki/Getblocktemplate

https://gitlab.com/bitcoin/eloipool/blob/master/gmp-proxy.py


Connect to custom bitcoind I have a fully synchronized btc node on my network.
Which mobile apps (android) and desktop apps can I use to connect to this node without running a full node/ daemon on the desktop or mobile? And how would I do that?
I really want to just transact through the local node. Every App I have looked at so far runs their own local full node.
The idea is that the family can transact through one local full-node.
Edit: should have read the cli command options on bitcoin-qt
A simple:
bitcoin-qt.exe -connect mynodeIp -addnode mynodeIp
Creates two connections to the node and blocks all others

Which mobile apps (android) and desktop apps can I use to connect to this node without running a full node/ daemon on the desktop or mobile?

Mobile apps:
Good UI/UX for using bitcoin core.
https://github.com/Fonta1n3/FullyNoded (iOS)
It's a simple app to connect to local node for broadcasting transaction. You can add more things if you are a developer.
https://github.com/percy-g2/BitK (Android)
Desktop apps:
https://github.com/cryptoadvance/specter-desktop
https://github.com/bwt-dev/bwt-electrum-plugin (Connect Electrum with bitcoin core full node)


What would happen if every miner adds a transaction to his block that did not occur? Let's say every single miner in the world added the same transaction to their block via trickery. The transaction was never initiated by the wallet holder. Would a block from these miners be added to the block chain?

If so what is the minimum number of miners needed to add a fake transaction to the block chain?

Only the wallet holder can create a valid transaction. That's what makes them the wallet holder. The holder of a wallet is the person who can create valid transactions to claim the funds in that wallet.

So the transaction is not valid. Any block containing an invalid transaction is itself invalid. So everyone else (exchanges, wallets, etcetera) would ignore that block and any blocks mined on top of it. So all these miners would lose out on the money they could have made by mining valid blocks.


Ripple testnet explorer This is Ripple main Explorer. But where can i find the transactions ledger from wallets on Ripple testnet.

There is a variant of viewing the transaction history at a certain address:
http://ripplerm.github.io/ripple-wallet/

Dont forget to switch the network to the test in the upper left corner. Good luck =)


Bitcoin client only has 1 connection? I have had bitcoin for a little bit now.  It has been going very smoothly.  I have set up 3 accounts for various activities, privacy, and security reasons.  I now can not get all of the block chain.  When I go to download more of the block chain it just sits at 175408.  I have been at it for hours.  When I first got bitcoin it worked perfectly, but now even after reinstalling the bas** it is still stuck there, and my total connections went from eight to a one.  If anybody has any suggestions on to what the problem could be I would be grateful.

Maybe the client is having a hard time finding peers.  If you close it and remove the addr.dat then restart, it will go through the dnsseed process again.

http://en.bitcoin.it/wiki/Data_directory


How many OP_RETURN transactions in the Blockchain would be too many? I realize this question may not have an answer and therefore might not be suitable for SE, but I'm very curious about OP_RETURN transactions.

OP_RETURN transactions appear to be the accepted mechanism for inserting metadata into the Blockchain. Considering the innumerable uses for metadata (contracts, colored coins, proof of ownership...), I foresee more and more non-monetary transactions filling up the Blockchain.

Will that be bad?

It is reasonable solution. The main problem is not in the size of blockcain itself. The problem is in growing the number of unspent outputs. OP_RETURN outputs do not increase size of Utxo database


What is the file wallet.dat.rewrite? In my %APPDATA%\Bitcoin folder (in Windows 7) I have a file called wallet.dat.rewrite

It was apparently last modified on 12th May 2011.

I don't think it's a file I created manually, so what is it?

Sometimes the wallet file must be rewritten from scratch entirely (in particular when encrypthing it), to prevent unencrypted material to remain in old never-overwritten areas of the file. To do this, bitcoin writes a new file (wallet.dat.rewrite) and then renames it to wallet.dat. If this process is interrupted, it's possible that the wallet.dat.rewrite file remains. You can delete it safely.


C-lightning Implementation password protect hsm while maintaing node access I was wondering if there was a way to encrypt the folder that contains the hsm_secret (and a lot of other operational files necessary for constant operation of a C-lightning node) without cutting it off from normal access required by an operational lightning network node?

In theory, the way that Lightning works allows for the use of a Hardware Security module to perform a lot of the logic which would allow for secret material to be stored externally on a secure device similar to a Trezor. However none exists at the time of writing, and persistent access to many of the files is likely required in the current software. The balance of a Lightning wallet should be for the moment considered to be an extremely hot wallet, rather than long term storage for this reason.


Sent bitcoins to cake address, then it pops into account for half an hour then gets sent to another address, even before the confirmations I got sent bitcoins to my cake address then the coins went into the account as unconfirmed for about half an hour then they were sent to another address, and the confirmations have finished and i dont have any coins in my account they are sitting in this other account which is similiar address to the one i sent to , so its a cake address, im wondering if i hit refresh bitcoin addy it will go to this new address that was created ? am i on the right track ?
cf5471592253c04cf12a6aba559a7aec531df42cee4a87f85968ce4187dddc16
anyhelp would be great

It looks like your money was stolen. Either the app is fake or broken or someone had access to your wallet at sometime (maybe they helped you set it up).
A blockchain explorer shows (most recent at top)
Transactions involving address

Hash: c423c3d9d4e72b16cbfb3a3c1a47c578a56f51bf00be4ecb07c588d957f7cf46

2021-05-21 01:01  55 confirmations  Privacy score: 55

Senders
  0.05796927 BTC    bc1qs69asfc0c0we5ynauzsfjtm7hwp6du6vs40srf

Recipients
   bc1q8tdu705hjqxlzar2mretdrcesuezkzphad2amc  0.05784046 BTC Unspent

Hash: cf5471592253c04cf12a6aba559a7aec531df42cee4a87f85968ce4187dddc16

2021-05-21 01:01 55 confirmations  Privacy score: 49

Senders
  0.01025588 BTC        35KKBAGr5ch6dwMYzrxW1oVdCvMHHoCuKn
  0.04729191 BTC        3QEUGgo3LsLFFqAV38q5bCxP3nH1phbyPi
  0.00069646 BTC        3JervgE4EoMhjbAYH11pQnsxTDQBeK7FKu

Recipients
  bc1qs69asfc0c0we5ynauzsfjtm7hwp6du6vs40srf

So money was sent in and a second transaction immediately removed it.
Just because both addresses start bc1 doesn't mean they are both Cakewallet addresses. Thats not how addresses work. Apart from the bc1 or 3 prefix, they are visually random.


Merkle Root and Merkle Proofs This is one of those questions that everyone seems to write about but the same bad explanations are just plagiarized from site to site.

Edit: The heart of my question is why Merkle trees are claimed to be efficient when determining if a leaf node is present or tampered with when only the root node hash and a leaf node hash value is known and when the Merkle hashes are not cached somewhere (which they appear not to be in the case of Bitcoin's transactions stored in a block)... I so far see only claims of efficiency not any credible demonstrations or explanations of same. The whole point, I understand, of the Merkle "proof" is that the hashes are stored and then the binary feature of the Merkle tree can be leveraged to "walk the tree" from leaf to root with only a fraction of hash calculations required; without the cached hashes, I submit that all hashes in the entire tree would need to be recalculated, as I explain below...

I understand how to build Merkle trees. What I don't understand are two key points about Merkle trees and one key point about Bitcoin's Merkle trees for transactions. For the sake of my questions below, consider this cited Merkle tree from investopedia.com:

Merkle Trees in General

People say that given the Merkle root hash and the hash to H(D), you can quickly determine:

A. Whether the H(D) leaf is in the Merkle tree.

B. Whether the H(D) leaf has been tampered with or whether it changed positions.

With respect to A: To do this, wouldn't you just look at the list of leaf hashes to determine whether H(D) is there? Why do anything with the Merkle root?

With respect to B: If you have the H(D) hash value and you know that its leaf pair is H(C), you can calculate H(CD). But at that point, if H(D) has changed and you know the old value for H(CD), wouldn't you immediately be able to short-circuit your investigation because the new H(CD) value would not equal the old H(CD) value? If you say to me that you don't have the old H(CD) value then I would ask you: How do I get the value to H(AB) then? It would seem that I need to recalculate the hash for H(AB). And if that's true, I also would need to recalculate the hash to H(EFGH), which means recalculating the entire right half of the Merkle tree starting at the leaf nodes. There isn't much efficiency in that.

My final question:

C. Does Bitcoin persist the Merkle tree hashes for not only the leaf nodes but for the entire tree all the way to the root? Where exactly and how exactly does it store this information? All I ever see is the Merkle root in the block... I think I once saw a Merkle value for a transaction node but I never saw any meta data related to non-tree Merkle nodes in the Bitcoin blockchain...

Thanks!

Reading your question, I suspect the cause for your confusion is around the setting in which Merkle trees are used. I think you are right that in the Bitcoin world, the setting is taken for granted and is not usually explained clearly.
Here's my two cents. To understand the efficiency aspect of Merkle trees, consider the two parties that are actually involved in the protocol:

A Bitcoin full node who has full blocks.
A Bitcoin thin node who has headers but wants to verify that a transaction is in one of the full blocks.
(It might be worth mentioning that the current Bitcoin protocol does not allow a thin node to efficiently verify that a transaction is not in any of the blocks.)

First, recall that when receiving a new block B with block header H and transactions T, the full node does the following:

computes h = SHA256(H), verifies the proof-of-work in h and checks the timestamp in H, etc.
for each transaction t \in T, verifies t's signature(s), verifies t does not double spend, and appends t as the rightmost leaf of the current block's Merkle tree. (Recall that a different Merkle tree is computed for each block. Of course, appending the transaction has to be done in a canonical order, so as to obtain the same root hash as in the block header H.)
after all t \in T have been verified and appended, computes and stores the Merkle tree, of course computing all interior node hashes and the root hash r (as depicted in your Merkle tree figure).
Crucially, verifies that the computed root hash r equals the root hash in the block header H.

Moral of the story: Yes, you are right that to prove membership of a leaf with respect to a Merkle root r, the prover (in this case, the full node) needs the interior node hashes: he can cache them or recompute them (there's some literature on trade-offs here). (I think) Bitcoin full nodes simply cache all hashes, as detailed in Step 3. Importantly, the interior hashes are not part of the 1MB block B: it would be a waste of space, since full nodes can simply recompute them from the leafs (i.e., the TXNs) and match the recomputed root hash with the one in the block header. Note that this prevents network adversaries from tampering with the transactions.
Second, recall that when receiving a new block header H, a thin node just does step (1) from above.
Now, here's the setting in Bitcoin:

The full node has all the blocks, including the interior node hashes computed in Step 3 above.
The thin node has all block headers. Recall that each header includes the Merkle root.
The full node wants to convince the thin node that some transaction t is in some block B with block header H (which of course the thin node has)

Thesis: Merkle trees allow the full node to convince the thin nodes of this efficiently, preventing the full node from changing t into t' (which would allow a full node to trick a thin node into thinking it got paid).
Proof: Let r denote the Merkle root in H. The thin node has that same H and r. The full node has the interior nodes "under" r cached (recall Step 3 from the full node). The full node fetches the (already cached) sibling path starting at the leaf containing the transaction t and going all the way up to the root r, sending it to the thin node. The thin node recursively hashes up t with the received siblings, obtaining a root hash r' (hand-waving here, assuming you understand Merkle proofs). The thin node checks that r == r'.
Efficiency: The full node (i.e., prover) only has to send O(\log{n}) siblings in a Merkle tree of n leafs to the thin node (i.e., the verifier), who only has a 256-bit root hash r. (Of course, the prover needs to have all the leafs and all the interior nodes, as discussed before.)
Naturally, this protocol guarantees the full node cannot "lie" about any transaction "under" the root r. Informally, if a transaction is there, the full node cannot change it to a different transaction and forge a proof for it (i.e., a sibling path). Also informally, if a transaction is not there at all, the full node cannot forge a proof for it being there.
To answer your questions:
Question A seems to be about how membership of a transaction t works. You say,

To do this, wouldn't you just look at the list of leaf hashes to determine whether H(D) is there? Why do anything with the Merkle root?

I think you're misunderstanding the setting. Keep in mind, that when discussing cryptographic protocols, it's quite necessary to refer to the parties by name, rather than using ambiguous pronouns like "you" and "I." "Prover" and "verifier" can be terse and boring, but is precise.
The thin node cannot "just look at the list of leaf hashes" because it only has the root hash and no leafs but wants to be sure that a leaf is there. The full node could send the thin node all the leafs and have the thin node hash them, obtaining the same root hash. However, that would be inefficient: O(n) bandwidth for n leafs. In contrast, a Merkle proof for a leaf under a root hash is much more efficient: only O(\log{n}) hashes + the leaf itself.
Question B should be answered by now: the full node has all interior node hashes. It sends the sibling path to the thin node, who simply recursively hashes up from the leaf being checked. The thin node verifies the root it obtained matches the root in the block header.
Question C:

Does Bitcoin persist the Merkle tree hashes for not only the leaf nodes but for the entire tree all the way to the root?

Not in the blocks themselves: it would be a waste of space as mentioned before. Full nodes compute the interior hashes and the root hash and cache everything locally on disk (presumably, I haven't actually looked at the source code). Then, they are ready to serve proofs for any transaction in any block to any thin node.

Where exactly and how exactly does it store this information? All I ever see is the Merkle root in the block...

Presumably, on disk in the database of the full node. But not in the block: they are redundant info because blocks contains leafs => can compute hashes of interior nodes and root node.

I think I once saw a Merkle value for a transaction node but I never saw any meta data related to non-tree Merkle nodes in the Bitcoin blockchain...

Ugh, I want to say "Yes, because it would be a waste of space to store interior node hashes in the blocks." but reading closer I don't actually know what you mean by "Merkle value for a transaction node" and "metadata related to non-tree Merkle nodes."
Hope this helps!