text:
The goal of this exercise is to implement DQN and to apply it to the cartpole balancing problem.

try:
IN_COLAB = True
except:
IN_COLAB = False

if IN_COLAB:
!pip install -U gymnasium pygame moviepy
!pip install gymnasium[box2d]
import numpy as np
rng = np.random.default_rng()
import matplotlib.pyplot as plt
import os
from IPython.display import clear_output
from collections import deque

import gymnasium as gym
print("gym version:", gym.__version__)

import pygame
from moviepy.editor import ImageSequenceClip, ipython_display

import tensorflow as tf
import logging
tf.get_logger().setLevel(logging.ERROR)

class GymRecorder(object):
"""
Simple wrapper over moviepy to generate a .gif with the frames of a gym environment.

The environment must have the render_mode rgb_array_list.
"""
def __init__(self, env):
self.env = env
self._frames = []

def record(self, frames):
"To be called at the end of an episode."
for frame in frames:
self._frames.append(np.array(frame))

def make_video(self, filename):
"Generates the gif video."
directory = os.path.dirname(os.path.abspath(filename))
if not os.path.exists(directory):
os.mkdir(directory)
del self._frames
self._frames = []

def running_average(x, N):
kernel = np.ones(N) / N
return np.convolve(x, kernel, mode='same')
gym version: 0.26.3

We are going to use the Cartpole balancing problem, which can be loaded with:

gym.make('CartPole-v0')

States have 4 continuous values (position and speed of the cart, angle and speed of the pole) and 2 discrete outputs (going left or right). The reward is +1 for each transition where the pole is still standing (angle of less than 30° with the vertical).

In CartPole-v0, the episode ends when the pole fails or after 200 steps. In CartPole-v1, the maximum episode length is 500 steps, which is too long for us, so we stick to v0 here.

The maximal (undiscounted) return is therefore 200. Can DQN learn this?

# Create the environment
env = gym.make('CartPole-v0', render_mode="rgb_array_list")
recorder = GymRecorder(env)

# Sample the initial state
state, info = env.reset()

# One episode:
done = False
return_episode = 0
while not done:

# Select an action randomly
action = env.action_space.sample()

# Sample a single transition
next_state, reward, terminal, truncated, info = env.step(action)

# End of the episode
done = terminal or truncated

# Update undiscounted return
return_episode += reward

# Go in the next state
state = next_state

print("Return:", return_episode)

recorder.record(env.render())
video = "videos/cartpole.gif"
recorder.make_video(video)
ipython_display(video)
Return: 19.0
MoviePy - Building file videos/cartpole.gif with imageio.

As the problem is quite simple (4 state variables, 2 actions), DQN can run on a single CPU. However, we advise that you run the notebook on a GPU in Colab to avoid emptying the battery of your laptop too fast or making it too warm as training takes quite a long time.

We will stop from now on to display the cartpole on colab, as we want to go fast.

## Creating the model

The first step is to create the value network using keras. We will not need anything fancy: a simple fully connected network with 4 input neurons, two hidden layers of 64 neurons each and 2 output neurons will do the trick. ReLU activation functions all along and the Adam optimizer.

Q: Which loss function should we use? Think about which arguments have to passed to model.compile() and what activation function is required in the output layer.

We will need to create two identical networks: the trained network and the target network. You should therefore create a method that returns a compiled model, so it can be called two times. You should pass it the environment (so the network can know how many input and output neurons it needs) and the learning rate for the Adam optimizer.

def create_model(env, lr):

model = Sequential()

# ...

return model

Q: Implement the method accordingly.

def create_model(env, lr):

model = tf.keras.models.Sequential()

print(model.summary())

return model

Let’s test this method by creating the trained and target networks.

Important: every time you call create_model, a new neural network will be instantiated but the previous ones will not be deleted. During this exercise, you may have to create hundreds of networks because of the incremental implementation of DQN: all networks will stay instantiated in the RAM, and your computer/colab tab will freeze after a while. Before creating new networks, delete all existing ones with:

tf.keras.backend.clear_session()

Q: Create the trained and target networks. The learning rate does not matter for now. Instantiate the Cartpole environment and print the output of both networks for the initial state (state, info = env.reset()). Are they the same?

Hint: model.predict(X, verbose=0) expects an array X of shape (N, 4), with N the number of examples. Here, we have only one example, so make sure to reshape state so it has the shape (1, 4) (otherwise tf will complain).

env = gym.make('CartPole-v0')

state, info = env.reset()
state = state.reshape((1, env.observation_space.shape[0]))
print("State:", state)

tf.keras.backend.clear_session()
trained_model = create_model(env, 0.001)
target_model = create_model(env, 0.001)

trained_prediction = trained_model.predict(state, verbose=0)[0]
target_prediction = target_model.predict(state, verbose=0)[0]

print('-'*10)
print("State:", state)
print("Prediction for the trained network:", trained_prediction)
print("Prediciton for the target network:", target_prediction)
print('-'*10)
State: [[0.01271655 0.00693787 0.00868186 0.00911883]]
Metal device set to: Apple M1 Pro
Model: "sequential"
_________________________________________________________________
Layer (type)                Output Shape              Param #
=================================================================
dense (Dense)               (None, 64)                320

dense_1 (Dense)             (None, 64)                4160

dense_2 (Dense)             (None, 2)                 130

=================================================================
Total params: 4,610
Trainable params: 4,610
Non-trainable params: 0
_________________________________________________________________
None
Model: "sequential_1"
_________________________________________________________________
Layer (type)                Output Shape              Param #
=================================================================
dense_3 (Dense)             (None, 64)                320

dense_4 (Dense)             (None, 64)                4160

dense_5 (Dense)             (None, 2)                 130

=================================================================
Total params: 4,610
Trainable params: 4,610
Non-trainable params: 0
_________________________________________________________________
None
2022-11-25 11:15:13.944109: I tensorflow/core/common_runtime/pluggable_device/pluggable_device_factory.cc:306] Could not identify NUMA node of platform GPU ID 0, defaulting to 0. Your kernel may not have been built with NUMA support.
2022-11-25 11:15:13.944323: I tensorflow/core/common_runtime/pluggable_device/pluggable_device_factory.cc:272] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 0 MB memory) -> physical PluggableDevice (device: 0, name: METAL, pci bus id: <undefined>)
2022-11-25 11:15:14.117562: W tensorflow/core/platform/profile_utils/cpu_utils.cc:128] Failed to get CPU frequency: 0 Hz
2022-11-25 11:15:14.148074: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:114] Plugin optimizer for device_type GPU is enabled.
----------
State: [[0.01271655 0.00693787 0.00868186 0.00911883]]
Prediction for the trained network: [0.00114096 0.00075357]
Prediciton for the target network: [-0.00087994  0.00059144]
----------
2022-11-25 11:15:14.260500: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:114] Plugin optimizer for device_type GPU is enabled.

The target network has the same structure as the trained network, but not the same weights, as they are randomly initialized. We want the target network \theta' to have exactly the same weights as the trained weights \theta. You can obtain the weights of a network with:

w = model.get_weights()

and set weights using:

model.set_weights(w)

Q: Transfer the weights of the trained model to the target model. Compare their predictions for the current state.

target_model.set_weights(trained_model.get_weights())

trained_prediction = trained_model.predict(state, verbose=0)[0]
target_prediction = target_model.predict(state, verbose=0)[0]

print('-'*10)
print("State:", state)
print("Prediction for the trained network:", trained_prediction)
print("Prediciton for the target network:", target_prediction)
print('-'*10)
----------
State: [[0.01271655 0.00693787 0.00868186 0.00911883]]
Prediction for the trained network: [0.00114096 0.00075357]
Prediciton for the target network: [0.00114096 0.00075357]
----------

## Experience replay memory

The second thing that we need is the experience replay memory (or replay buffer). We need a container like a python list where we append (s, a, r, s’, done) transitions (as in Q-learning), but with a maximal capacity: when there are already C transitions in the list, one should stop appending to the list, but rather start writing at the beginning of the list.

This would not be very hard to write, but it would take a lot of time and the risk is high to have hard-to-notice bugs.

Here is a basic implementation of the replay buffer using double-ended queues (deque). A deque is list with a maximum capacity. If the deque is full, it starts writing again at the beginnning. Exactly what we need. This implementation uses one deque per element in (s, a, r, s’, done), but one could also append the whole transition to a single deque.

Q: Read the code of the ReplayBuffer and understand what it does.

class ReplayBuffer:
"Basic implementation of the experience replay memory using separated deques."
def __init__(self, max_capacity):
self.max_capacity = max_capacity

# deques for each element
self.states = deque(maxlen=max_capacity)
self.actions = deque(maxlen=max_capacity)
self.rewards = deque(maxlen=max_capacity)
self.next_states = deque(maxlen=max_capacity)
self.dones = deque(maxlen=max_capacity)

def append(self, state, action, reward, next_state, done):
# Store data
self.states.append(state)
self.actions.append(action)
self.rewards.append(reward)
self.next_states.append(next_state)
self.dones.append(done)

def sample(self, batch_size):
# Do not return samples if we do not have at least 2*batch_size transitions
if len(self.states) < 2*batch_size:
return []

# Randomly choose the indices of the samples.
indices = sorted(np.random.choice(np.arange(len(self.states)), batch_size, replace=False))

# Return the corresponding
return [np.array([self.states[i] for i in indices]),
np.array([self.actions[i] for i in indices]),
np.array([self.rewards[i] for i in indices]),
np.array([self.next_states[i] for i in indices]),
np.array([self.dones[i] for i in indices])]

Q: Run a random agent on Cartpole (without rendering) for a few episodes and append each transition to a replay buffer with small capacity (e.g. 100). Sample a batch to check that everything makes sense.

env = gym.make('CartPole-v0')

buffer = ReplayBuffer(100)

for episode in range(10):

# Reset
state, info = env.reset()
done = False

# Sample the episode
while not done:

# Select an action randomly
action = env.action_space.sample()

# Perform the action
next_state, reward, terminal, truncated, info = env.step(action)

# End of the episode
done = terminal or truncated

# Store the transition
buffer.append(state, action, reward, next_state, done)

# Go in the next state
state = next_state

# Sample a minibatch
batch = buffer.sample(10)
print("States:", batch[0])
print("Actions:", batch[1])
print("Rewards:", batch[2])
print("Next states:", batch[3])
print("Dones:", batch[4])
States: [[-7.1889226e-05 -1.9597625e-02  1.1202861e-02  1.0221607e-02]
[-5.1598279e-03 -2.1528986e-01  1.7882740e-02  3.1551802e-01]
[-7.8443229e-02 -8.0612069e-01  1.3144340e-01  1.3234164e+00]
[-4.6816234e-02  2.0717475e-01  4.1258741e-02 -2.8127652e-01]
[-4.2442955e-02 -1.8412507e-01  3.6115777e-02  3.2783771e-01]
[-9.1748878e-02 -3.8812301e-01  1.3098954e-01  8.2237107e-01]
[ 7.3674910e-02  6.0291737e-01 -7.6641589e-02 -8.1436384e-01]
[ 8.5733257e-02  7.9900050e-01 -9.2928864e-02 -1.1301358e+00]
[ 1.2985145e-01  6.0852367e-01 -1.5678419e-01 -9.4639248e-01]
[ 1.5025000e-02  7.9516947e-01 -4.6248329e-03 -1.1131814e+00]]
Actions: [0 0 1 0 0 0 1 0 0 1]
Rewards: [1. 1. 1. 1. 1. 1. 1. 1. 1. 1.]
Next states: [[-4.6384174e-04 -2.1487844e-01  1.1407293e-02  3.0641800e-01]
[-9.4656255e-03 -4.1066191e-01  2.4193101e-02  6.1378646e-01]
[-9.4565637e-02 -6.1288112e-01  1.5791173e-01  1.0745906e+00]
[-4.2672738e-02  1.1489304e-02  3.5633210e-02  2.4128472e-02]
[-4.6125453e-02 -3.7974206e-01  4.2672534e-02  6.3168758e-01]
[-9.9511340e-02 -5.8477044e-01  1.4743695e-01  1.1532161e+00]
[ 8.5733257e-02  7.9900050e-01 -9.2928864e-02 -1.1301358e+00]
[ 1.0171327e-01  6.0521013e-01 -1.1553158e-01 -8.6798614e-01]
[ 1.4202191e-01  4.1582093e-01 -1.7571205e-01 -7.0678967e-01]
[ 3.0928390e-02  9.9035186e-01 -2.6888460e-02 -1.4073114e+00]]
Dones: [False False False False False False False False False False]

## DQN agent

Here starts the fun part. There are a lot of things to do here, but you will now whether it works or not only when everything has been (correctly) implemented. So here is a lot of text to read carefully, and then you are on your own.

Reminder from the lecture:

• Initialize value network Q_{\theta} and target network Q_{\theta'}.

• Initialize experience replay memory \mathcal{D} of maximal size N.

• for t \in [0, T_\text{total}]:

• Select an action a_t based on Q_\theta(s_t, a), observe s_{t+1} and r_{t+1}.

• Store (s_t, a_t, r_{t+1}, s_{t+1}) in the experience replay memory.

• Every T_\text{train} steps:

• Sample a minibatch \mathcal{D}_s randomly from \mathcal{D}.

• For each transition (s_k, a_k, r_k, s'_k) in the minibatch:

• Compute the target value t_k = r_k + \gamma \, \max_{a'} Q_{\theta'}(s'_k, a') using the target network.
• Update the value network Q_{\theta} on \mathcal{D}_s to minimize:

\mathcal{L}(\theta) = \mathbb{E}_{\mathcal{D}_s}[(t_k - Q_\theta(s_k, a_k))^2]

• Every T_\text{target} steps:

• Update target network: \theta' \leftarrow \theta.

Here is the skeleton of the DQNAgent class that you have to write:

class DQNAgent:

def __init__(self, env, create_model, some_parameters):

self.env = env

# TODO: copy the parameters

# TODO: Create the trained and target networks, copy the weights.

# TODO: Create an instance of the replay memory

def act(self, state):

# TODO: Select an action using epsilon-greedy on the output of the trained model

return action

def update(self, batch):

# TODO: train the model using the batch of transitions

return loss # mse on the batch

def train(self, nb_episodes):

returns = []
losses = []

# TODO: Train the network for the given number of episodes

return returns, losses

def test(self):

# TODO: one episode with epsilon temporarily set to 0

return nb_steps # Should be 200 after learning

With this structure, it will be very simple to actually train the DQN on Cartpole:

# Create the environment
env = gym.make('CartPole-v1')

# Create the agent
agent = DQNAgent(env, create_model, other_parameters)

# Train the agent
returns, losses = agent.train(nb_episodes)

# Plot the returns
plt.figure(figsize=(10, 6))
plt.plot(returns)
plt.plot(running_mean(returns, 10))
plt.xlabel("Episodes")
plt.ylabel("Returns")

# Plot the losses
plt.figure(figsize=(10, 6))
plt.plot(losses)
plt.xlabel("Episodes")
plt.ylabel("Training loss")

plt.show()

# Test the network
nb_steps = agent.test()
print("Number of steps:", nb_steps)

So you “just” have to fill the holes.

### 1 - __init__(): Initializing the agent

In this method, you should first copy the value of the parameters as attributes: learning rate, epsilon, gamma and so on.

Suggested values: gamma = 0.99, learning_rate = 0.001

The second thing to do is to create the trained and target networks (with the same weights) and save them as attributes (the other methods will use them). Do not forget to clear the keras session first, otherwise the RAM will be quickly filled.

The third thing is to create an instance of the ERM. Use a buffer limit of 5000 transitions (should be passed as a parameter).

Do not hesitate to add other stuff as you implementing the other methods (e.g. counters).

### 2 - act(): action selection

We will use a simple \epsilon-greedy method for the action selection, as in the previous exercises.

The only difference is that we have to use the trained model to get the greedy action, using trained_model.predict(X, verbose=0). This will return the Q-value of the two actions left and right. Use argmax() to return the greedy action (with probability 1 - \epsilon). env.action_space.sample() should be used for the exploration (do not use the Q-network in that case, it is slow!).

\epsilon will be scheduled with an initial value of 1.0 and an exponential decay rate of 0.0005 after each action. It is always better to keep a little exploration, even if \epsilon has decayed to 0. Keep a minimal value of 0.05 for epsilon.

Q: Once this has been implemented, run your very slow random agent for 100 episodes to check everything works correctly.

### 3 - train(): training loop

This method will be very similar to the Q-learning agent that you implemented previously. Do not hesitate to copy and paste.

Here is the parts of the DQN algorithm that should be implemented:

• for t \in [0, T_\text{total}]:

• Select an action a_t based on Q_\theta(s_t, a), observe s_{t+1} and r_{t+1}.

• Store (s_t, a_t, r_{t+1}, s_{t+1}) in the experience replay memory.

• Every T_\text{train} steps:

• Sample a minibatch \mathcal{D}_s randomly from \mathcal{D}.

• Update the trained network using \mathcal{D}_s.

• Every T_\text{target} steps:

• Update target network: \theta' \leftarrow \theta.

The main difference with Q-learning is that update() will be called only every T_train = 4 steps: the number of updates to the trained network will be 4 times smaller that the number of steps made in the environment. Beware that if the ERM does not have enough transitions yet (less than the batch size), you should not call update().

Updating the target network (copying the weights of the trained network) should happen every 100 steps. Pass these parameters to the constructor of the agent.

The batch size can be set to 32.

### 4 - update(): training the value network

Using the provided minibatch, one should implement the following part of the DQN algorithm:

• For each transition (s_k, a_k, r_k, s'_k) in the minibatch:

• Compute the target value t_k = r_k + \gamma \, \max_{a'} Q_{\theta'}(s'_k, a') using the target network.
• Update the value network Q_{\theta} on \mathcal{D}_s to minimize:

\mathcal{L}(\theta) = \mathbb{E}_{\mathcal{D}_s}[(t_k - Q_\theta(s_k, a_k))^2]

So we just need to define the targets for each transition in the minibatch, and call model.fit() on the trained network to minimize the mse between the current predictions Q_\theta(s_k, a_k) and the target.

But we have a problem: the network has two outputs for the actions left and right, but we have only one target for the action that was executed. We cannot compute the mse between a vector with 2 elements and a single value… They must have the same size.

As we want only the train the output neuron corresponding to the action a_k, we are going to:

1. Use the trained network to predict the Q-value of both actions [Q_\theta(s_k, 0), Q_\theta(s_k, 1)].
2. Replace one of the values with the target, for example [Q_\theta(s_k, 0), t_k] if the second action was chosen.
3. Minimize the mse between [Q_\theta(s_k, 0), Q_\theta(s_k, 1)] and [Q_\theta(s_k, 0), t_k].

That way, the first output neuron has a squared error of 0, so it won’t learn anything. Only the second output neuron will have a non-zero mse and learn.

There are more efficient ways to do this (using masks), but this will do the trick, the drawback being that we have to make a forward pass on the minibatch before calling fit().

The rest is pretty much the same as for your Q-learning agent. Do not forget that actions leading to a terminal state should only use the reward as a target, not the complete Bellman target r + \gamma \max Q.

Hint: as we sample a minibatch of 32 transitions, it is faster to call:

Q_values = np.array(training_model.predict_on_batch(states))

than:

Q_values = training_model.predict(states)

for reasons internal to tensorflow. Note that with tf2, you need to cast the result to numpy arrays as eager mode is now the default.

The method should return the training loss, which is contained in the History object returned by model.fit(). model.fit() should be called for one epoch only, a batch size of 32, and verbose set to 0.

### 5 - test()

This method should run one episode with epsilon set to 0, without learning. The number of steps should be returned (do not bother discounting with gamma, the goal is to be up for 200 steps).

Q: Let’s go! Run the agent for 150 episodes and observe how fast it manages to keep the pole up for 200 steps.

Beware that running the same network twice can lead to very different results. In particular, policy collapse (the network was almost perfect, but suddenly crashes and becomes random) can happen. Just be patient.

You can visualize a test trial using the GymRecorder: you just need to set the env attribute of your DQN agent to a new env with the render mode rgb_array_list and record the frames at the end.

class DQNAgent:

def __init__(self, env, create_model, learning_rate, epsilon, epsilon_decay, gamma, batch_size, target_update_period, training_update_period, buffer_limit):
self.env = env

self.learning_rate = learning_rate
self.epsilon = epsilon
self.epsilon_decay = epsilon_decay
self.gamma = gamma
self.batch_size = batch_size
self.target_update_period = target_update_period
self.training_update_period = training_update_period

# Create the Q-network and the target network
tf.keras.backend.clear_session() # start by deleting all existing models to be gentle on the RAM
self.model = create_model(self.env, self.learning_rate)
self.target_model = create_model(self.env, self.learning_rate)
self.target_model.set_weights(self.model.get_weights())

# Create the replay memory
self.buffer = ReplayBuffer(buffer_limit)

def act(self, state):

# epsilon-greedy
if np.random.rand() < self.epsilon: # Random selection
action = self.env.action_space.sample()
else: # Use the Q-network to get the greedy action
action = self.model.predict(state.reshape((1, env.observation_space.shape[0])), verbose=0)[0].argmax()

# Decay epsilon
self.epsilon *= 1 - self.epsilon_decay
self.epsilon = max(0.05, self.epsilon)

return action

def update(self, batch):

# Get the minibatch
states, actions, rewards, next_states, dones = batch

# Predict the Q-values in the current state
targets = np.array(self.model.predict_on_batch(states))

# Predict the Q-values in the next state using the target model
next_Q_value = np.array(self.target_model.predict_on_batch(next_states)).max(axis=1)

# Terminal states have a value of 0
next_Q_value[dones] = 0.0

# Compute the target
for i in range(self.batch_size):
targets[i, actions[i]] = rewards[i] + self.gamma * next_Q_value[i]

# Train the model on the minibatch
history = self.model.fit(states, targets, epochs=1, batch_size=self.batch_size, verbose=0)

return history.history['loss'][0]

def train(self, nb_episodes):

steps = 0
returns = []
losses = []

for episode in range(nb_episodes):

# Reset
state, info = self.env.reset()
done = False
steps_episode = 0
return_episode = 0

loss_episode = []

# Sample the episode
while not done:

# Select an action
action = self.act(state)

# Perform the action
next_state, reward, terminal, truncated, info = self.env.step(action)

# End of the episode
done = terminal or truncated

# Store the transition
self.buffer.append(state, action, reward, next_state, done)

# Sample a minibatch
batch = self.buffer.sample(self.batch_size)

# Train the NN on the minibatch
if len(batch) > 0 and steps % self.training_update_period == 0:
loss = self.update(batch)
loss_episode.append(loss)

# Update the target model
if steps > self.target_update_period and steps % self.target_update_period == 0:
self.target_model.set_weights(self.model.get_weights())

# Go in the next state
state = next_state

# Increment time
steps += 1
steps_episode += 1
return_episode += reward

if done:
break

# Store info
returns.append(return_episode)
losses.append(np.mean(loss_episode))

# Print info
clear_output(wait=True)
print('Episode', episode+1)
print(' total steps:', steps)
print(' length of the episode:', steps_episode)
print(' return of the episode:', return_episode)
print(' current loss:', np.mean(loss_episode))
print(' epsilon:', self.epsilon)

return returns, losses

def test(self, render=True):

old_epsilon = self.epsilon
self.epsilon = 0.0

state, info = self.env.reset()
nb_steps = 0
done = False

while not done:
action = self.act(state)
next_state, reward, terminal, truncated, info = self.env.step(action)
done = terminal or truncated
state = next_state
nb_steps += 1

self.epsilon = old_epsilon
return nb_steps
# Parameters
nb_episodes = 150
batch_size = 32

epsilon = 1.0
epsilon_decay = 0.0005

gamma = 0.99

learning_rate = 0.005
buffer_limit = 5000
target_update_period = 100
training_update_period = 4
# Create the environment
env = gym.make('CartPole-v0')

# Create the agent
agent = DQNAgent(env, create_model, learning_rate, epsilon, epsilon_decay, gamma, batch_size, target_update_period, training_update_period, buffer_limit)

# Train the agent
returns, losses = agent.train(nb_episodes)

# Plot the returns
plt.figure(figsize=(10, 6))
plt.plot(returns)
plt.plot(running_average(returns, 10))
plt.xlabel("Episodes")
plt.ylabel("Returns")

# Plot the losses
plt.figure(figsize=(10, 6))
plt.plot(losses)
plt.xlabel("Episodes")
plt.ylabel("Training loss")
plt.show()
Episode 150
total steps: 17021
length of the episode: 138
return of the episode: 138.0
current loss: 7.081971720286778
epsilon: 0.05
# Test the network
env = gym.make('CartPole-v0', render_mode="rgb_array_list")
recorder = GymRecorder(env)
agent.env = env

nb_steps = agent.test()
recorder.record(env.render())
print("Number of steps:", nb_steps)

video = "videos/cartpole-dqn.gif"
recorder.make_video(video)
ipython_display(video, loop=0, autoplay=1)
Number of steps: 162
MoviePy - Building file videos/cartpole-dqn.gif with imageio.

Q: How does the loss evolve? Does it make sense?

A: The Q-values are non-stationary: the initial Q-values are very small (the agent fails almost immediately), while they are around 100 after training (200 steps with a reward of +1, but discounted with gamma). The mse increases with the magnitude of the Q-values, so the loss is a poor indicator of the convergence of the network.

## Reward scaling

Q: Do a custom test trial after training (i.e. do not call test(), but copy and adapt its code) and plot the Q-value of the selected action at each time step. Do you think it is a good output for the network? Could it explain why learning is so slow?

env = gym.make('CartPole-v0')
agent.env = env

# No exploration
agent.epsilon = 0.0

state, info = agent.env.reset()
done = False

Q_values = []

while not done:
action = agent.act(state)
Q_values.append(agent.model.predict(state.reshape((1, 4)), verbose=0)[0][action])
next_state, reward, terminal, truncated, info = agent.env.step(action)
done = terminal or truncated
state = next_state

plt.figure(figsize=(10, 6))
plt.plot(Q_values)
plt.xlabel("Steps")
plt.ylabel("Q-value")
plt.show()

A: The predicted Q-values at the beginning of learning are close to 0, as the weights are randomly initialized. They should grow to around 100, which takes a lot of time. If the target Q-values were around 1, learning might be much faster.

Q: Implement reward scaling by dividing the received rewards by a fixed factor of 100 when computing the Bellman targets. That way, the final Q-values will be around 1, what may be much easier to learned.

Tip: in order to avoid a huge copy and paste, you can inherit from your DQNAgent and only reimplement the desired function:

class ScaledDQNAgent (DQNAgent):
def update(self, batch):
# Change the content of this function only

You should reduce a bit the learning rate (e.g. 0.001) as the magnitude of the targets has changed.

class ScaledDQNAgent(DQNAgent):

def update(self, batch):

# Get the minibatch
states, actions, rewards, next_states, dones = batch

# Predict the Q-values in the current state
targets = np.array(self.model.predict_on_batch(states))

# Predict the Q-values in the next state using the target model
next_Q_value = np.array(self.target_model.predict_on_batch(next_states)).max(axis=1)

# Terminal states have a value of 0
next_Q_value[dones] = 0.0

# Compute the target
for i in range(self.batch_size):
targets[i, actions[i]] = rewards[i]/100. + self.gamma * next_Q_value[i]

# Train the model on the minibatch
history = self.model.fit(states, targets, epochs=1, batch_size=self.batch_size, verbose=0)

return history.history['loss'][0]
# Create the environment
env = gym.make('CartPole-v0')

# Create the agent
learning_rate = 0.002
agent = ScaledDQNAgent(env, create_model, learning_rate, epsilon, epsilon_decay, gamma, batch_size, target_update_period, training_update_period, buffer_limit)

# Train the agent
returns, losses = agent.train(nb_episodes)

# Plot the returns
plt.figure(figsize=(10, 6))
plt.plot(returns)
plt.plot(running_average(returns, 10))
plt.xlabel("Episodes")
plt.ylabel("Returns")

# Plot the losses
plt.figure(figsize=(10, 6))
plt.plot(losses)
plt.xlabel("Episodes")
plt.ylabel("Training loss")
Episode 150
total steps: 18540
length of the episode: 200
return of the episode: 200.0
current loss: 0.0003936138337303419
epsilon: 0.05
Text(0, 0.5, 'Training loss')
# Test the network
env = gym.make('CartPole-v0', render_mode="rgb_array_list")
recorder = GymRecorder(env)
agent.env = env

agent.epsilon = 0.0

# Q-values
state, info = agent.env.reset()
done = False
Q_values = []

while not done:
action = agent.act(state)
Q_values.append(agent.model.predict(state.reshape((1, 4)), verbose=0)[0][action])
next_state, reward, terminal, truncated, info = agent.env.step(action)
done = terminal or truncated
state = next_state

# Plot the Q-values
plt.figure(figsize=(10, 6))
plt.plot(Q_values)
plt.xlabel("Steps")
plt.ylabel("Q-value")
plt.show()
recorder.record(env.render())

video = "videos/cartpole-dqn-scaled.gif"
recorder.make_video(video)
ipython_display(video, loop=0, autoplay=1)
MoviePy - Building file videos/cartpole-dqn-scaled.gif with imageio.


text:
# Leetcode 326 - Power of Three

#### Posted on February 28, 2016

##### Last updated on March 2, 2016

Problem link. This is simple, just make sure you don’t overflow (I do it with the q > 0 check).

Markdown SHA1: f4e05e8562a4f880d78aedc1e313a622f69f546b

format: plain


text:
# Centralized Critic PPO¶

TL;DR

A PPO agent using a Centralized Critic. The implementation uses a multi-agent variation of Centralized Critic with a Set Transformer for preprocessing the critic’s input.

## 💡 The idea¶

The Flatland environment presents challenges to learning algorithms with respect to the agent’s behavior when faced with the presence of other agents, as well as with sparse and delayed rewards. Centralized critic methods are a way to deal with such problematic multi-agent training situations.

The base architecture implemented here is a a fully connected network with PPO trainer. At execution time the agents will step through the environment in the usual way. During training, however, a different network, is used that provides the gradients on which to train the agent’s network. This “central critic” is provided with a more holistic view of the simulation state in order to better approximate an adequate reward for the agent’s actions in the current situation.

Different variations of centralized critics exist. The method contained in the baselines does not use global observations as many other implementations, but combines the observations of the to-be-trained-agent with the observations of the other agents. The approach was chosen in order to create a solution that would be feasible in large environments. This might not be the solution with the best performance and participants should therefore not be deterred from developing more specialized variations.

The model also contains a Set Transformer. This transformer sits between the observations of the agents and the centralized critic in order to, providing the critic with an embedding of the observations. This is useful for dealing with the sparsity of the observations, with the temporal nature of the data, as well as with possible permutation invariance in the case of a changing subset of agents included in the observation.

## 🗂 Files and usage¶

The general structure of the code allows substitution of the separate parts, which will be discussed in the next section.

### 🚂 Training¶

An example configuration file can be found in neurips2020-flatland-baselines/experiments/flatland_sparse/small_v0/tree_obs_fc_net/ccppo.yaml.

Run it with:

\$ python ./train.py -f experiments/flatland_sparse/small_v0/tree_obs_fc_net/ccppo.yaml

### 🧠 Model¶

The model is implemented in neurips2020-flatland-baselines/models/cc_transformer.py

## 📦 Implementation Details¶

The cc_transformer model file contains the code for all concepts needed for central-critic and transformer implementation. The three components Transformer, Agent-Model and Critic-Model are constructed and put together in the CentralizedCriticModel class.

### Transformer¶

The transformer is built with self-attention layers which can be found in the MultiHeadAttentionLayer class. The layers will calculate the embedding with trained linear projections. The dimensions of these outputs can be decided on, as they are the product of the number of parallel heads (num_head) and the output dimension (d_model).

### Central Critic¶

The central critic is constructed with the build_fullyConnected function to build a fully connected network of the given input, output and hidden layer sizes.

### Agent Model¶

The agents are built with fully connected layers in the build_fullyConnected to build a fully connected network of the given input, output and hidden layer sizes.

### Trainer¶

For training the fully connected layers we use the standard PPO trainer implementation provided by RLlib with necessary updates to the post-processing. In centralized_critic_postprocessing we ensure that training_batches contain all the necessary observations of neighboring agents, as well as performing the advantage estimation.

The trainer registration is done outside of classes or functions at the end of the file. Here, the CCPPO Policy is defined, based on the standard PPOTrainer and subsequently used to register the model to be used in ray.

## 📈 Results¶

The described approach was developed and evaluated in a project of Deutsche Bahn in cooperation with InstaDeep. In the context of the project CCPPO was validated and directly compared to standard PPO on a previous version of Flatland. This original implementation that used a custom version of a sparse reward and a custom observation similar to Flatland’s tree observation outperformed standard PPO. For the current baselines the CCPPO model was run using the new Flatland version together with its standard observation and reward. The graph below shows the results for these runs with and without the transformer. For comparison, a run of the current best method in the baseline (Ape-X) with comparable sizes and parameters is shown. The full configuration can be found in the corresponding CCPPO config files. The results show that CCPPO is able to learn in this setup, but with a lower performance than Ape-X. We provide an example setup for experiments with CCPPO and encourage participants to further refine and develop this approach.


text:
# IEEE Transactions on Instrumentation and Measurement

## Filter Results

Displaying Results 1 - 25 of 25
• ### Embedded instrumentation for evaluating task sharing performance in a distributed computing system

Publication Year: 1992, Page(s):316 - 321
| | PDF (536 KB)

An example of application-level performance measurement on a distributed computing system using software-based instrumentation is considered. Embedded software instrumentation for measuring the performance of a decentralized adaptive task-sharing algorithm in a distributed computing system is described. For the application considered, the simplicity and flexibility provided by software-based instr... View full abstract»

• ### Measurement problems arising from the use of a recursive algorithm for model identification of electrical systems

Publication Year: 1992, Page(s):240 - 245
Cited by:  Papers (18)
| | PDF (488 KB)

The use of parameter estimation techniques in practical applications requires accurate analysis of the associated measurement and computation problems. With reference to an already proposed model identification procedure, the authors deal with the experimental tests carried out in order to highlight problems and to find the most appropriate solutions. In particular, a synchronization method is des... View full abstract»

• ### New method for reactive power and energy measurement

Publication Year: 1992, Page(s):280 - 285
Cited by:  Papers (19)
| | PDF (420 KB)

The properties of an electronic reactive power and energy measurement method based on a frequency-controlled power-to-pulse-rate conversion are analyzed. The reactive power and energy measurements can be efficiently performed by using a simple integrating circuit as a phase shifter, and frequency-controlled analog-to-pulse-rate converter. A low value of the output pulse rate should not be consider... View full abstract»

• ### Optimization and analysis of a capacitive contactless angular transducer

Publication Year: 1992, Page(s):311 - 315
Cited by:  Papers (7)
| | PDF (284 KB)

A capacitive angular transducer under development for industrial and military applications is described. An operational description based on the physical configuration of the transducer is presented in terms of a lumped parameter model. From the model, analytical operational performance criteria, i.e. frequency response and accuracy, are developed, analyzed, and supported with experimental data. M... View full abstract»

• ### Parametric identification of two-port models in the frequency domain

Publication Year: 1992, Page(s):233 - 239
Cited by:  Papers (18)  |  Patents (1)
| | PDF (508 KB)

The authors treat the problem of parametric estimation of linear time-invariant dynamic two-port models (e.g. the short-circuit admittance matrix) from experimental data. A multivariate frequency-domain Gaussian maximum likelihood estimator is proposed to estimate the unknown coefficients occurring in the rational two-port model. It takes the perturbing noise of all the measured voltages and curre... View full abstract»

• ### Simultaneous measurements of dynamic values using the transit time method

Publication Year: 1992, Page(s):251 - 255
Cited by:  Papers (1)
| | PDF (352 KB)

The problem of simultaneous and accurate measurements of two dynamic values, time dependencies of flow velocity and ultrasound velocity in the flow, is analyzed. In order to measure two dynamic values simultaneously a theory of the transit time method has developed, and the theoretical model of a microprocessor-based measuring system has been derived. The ways to improve the accuracy and informati... View full abstract»

• ### Process data chemometrics

Publication Year: 1992, Page(s):262 - 268
Cited by:  Papers (51)  |  Patents (3)
| | PDF (608 KB)

A data analysis technique successfully applied in spectroscopy to reduce a large quantity of data into meaningful information, called chemometrics, is discussed. Data when properly interpreted by statistical data analysis tools and fundamental and heuristic models yield meaningful information. The use of chemometrics as a multivariate analyzer to provide a composite measurement of the state of a c... View full abstract»

• ### Both active and reactive power and energy transfer standard

Publication Year: 1992, Page(s):274 - 279
Cited by:  Papers (7)
| | PDF (580 KB)

A thermal converter to DC voltage, with 10-ppm accuracy and a frequency range of 0-200 kHz, which can be used for active power measurements, has been developed using an infrared thermal converter. Moreover, a proposal for implementing, with lower accuracy, the converter for energy and reactive power is also illustrated View full abstract»

• ### Automated swept-angle bistatic scattering measurements using continuous wave radar

Publication Year: 1992, Page(s):185 - 192
Cited by:  Papers (25)
| | PDF (648 KB)

The establishment of the IEEE Electromagnetic Modeling Software Committee and the Department of Defense sponsored Electromagnetic Code Consortium indicates the electromagnetics community is interested in validating general electromagnetic modeling software. An important part of this validation is obtaining reliable bistatic scattering measurements on canonical scatterers. The author describes a si... View full abstract»

• ### A potentiostat based on a voltage-controlled current source for use with amperometric gas sensors

Publication Year: 1992, Page(s):308 - 310
Cited by:  Papers (16)
| | PDF (208 KB)

Amperometric gas-sensing devices develop a current that is proportional to the concentration of gas that is supplied to them. The circuit that maintains electrochemical stability in the sensor as well as buffering the current output is called a potentiostat. Some sensors used for gas sensing have very large electrode capacitances, which can result in instability with conventional potentiostat desi... View full abstract»

• ### The interpolated fast Fourier transform: a comparative study

Publication Year: 1992, Page(s):226 - 232
Cited by:  Papers (200)  |  Patents (2)
| | PDF (472 KB)

The properties of five interpolating fast Fourier transform (IFFT) methods (IFFT) are studied with respect to their systematic errors and their noise sensitivity, for a monofrequency signal. It is shown that windows with small spectral side lobes do not always result in a better overall performance of the IFFT method and that time-domain estimators can be more efficient than the analyzed IFFT meth... View full abstract»

• ### Filtering capabilities and convergence of the Van-Cittert deconvolution technique

Publication Year: 1992, Page(s):246 - 250
Cited by:  Papers (21)
| | PDF (412 KB)

The authors demonstrate the application of the Van-Cittert technique in iterative frequency-domain deconvolution. The technique is shown to have built-in filtering capabilities which can be used successfully to produce optimum deconvolution estimates. The Bennia-Riad optimization criterion for iterative deconvolution is jointly used with the Van-Cittert technique to optimize the number of iteratio... View full abstract»

• ### Measurement and analysis for stripline material parameters using network analyzers

Publication Year: 1992, Page(s):286 - 290
Cited by:  Papers (1)
| | PDF (360 KB)

The characterization of material parameters for dielectrics used in the construction of stripline and related structures is presented. The measured scattering parameters of transmission-line samples are used to predict the characteristics of the dielectric materials used in the construction in addition to effects due to rough dielectric-conductor interfaces. The measurements have been performed us... View full abstract»

• ### The influence of windowing on the accuracy of multifrequency signal parameter estimation

Publication Year: 1992, Page(s):256 - 261
Cited by:  Papers (115)
| | PDF (480 KB)

The author investigates the influence of windowing on the accuracy of a frequency-domain procedure suitable for real-time estimations of multifrequency signal parameters. Imposing very weak assumptions about the statistical description of the noise, simple and accurate expressions for the estimator variances are established. They allow both the evaluation of the result accuracy and the design of t... View full abstract»

• ### Calibration of an electronic multisensor system for measuring high pressures

Publication Year: 1992, Page(s):269 - 273
Cited by:  Papers (14)
| | PDF (336 KB)

An electronic system for measuring high pressures is considered. Its functioning is based on three fiber-optic transducers whose static characteristics are nonmonotonic and significantly influenced by temperature. A method for static calibration of this system is proposed, based on the use of spline functions of one variable. Its applicability in the considered case is demonstrated using synthetic... View full abstract»

• ### Analysis of symmetrical six-port junction when configured as a six-port reflectometer

Publication Year: 1992, Page(s):193 - 197
Cited by:  Papers (15)
| | PDF (340 KB)

The symmetrical six-port junction (together with a directional coupler and four power detectors) has in the ideal case been shown to be suitable for use as a six-port reflectometer. The author presents formulas that can predict the performance of such a reflectometer when the symmetrical six-port junction is nonideal. The results obtained indicate that in the presence of junction imperfections, it... View full abstract»

• ### EMI shielding measurements of conductive polymer blends

Publication Year: 1992, Page(s):291 - 297
Cited by:  Papers (258)  |  Patents (1)
| | PDF (628 KB)

Shielding efficiencies for a number of blends containing a variety of conductive fillers, including intrinsically conductive polymers, have been measured in the near field with a dual-chamber box and in the far field with a transmission line fixture. Although all samples satisfied the classical good-conductor approximation, most of them exhibited a crossover from being electrically thin (thickness... View full abstract»

• ### Video display terminal fringing magnetic field measurements

Publication Year: 1992, Page(s):178 - 184
Cited by:  Papers (3)
| | PDF (588 KB)

Possible occupational health or public health risks due to video display terminals (VDTs) are addressed. The authors note the errors of current jargon and correctly define the terms needed to describe the fringing magnetic flux density near any magnetically deflected, raster scan, cathode ray tube display. Theory and preliminary measurements have shown that the ELF AC electric field related to VDT... View full abstract»

• ### Scale-model design of a submillimeter-wave planar diode mixer

Publication Year: 1992, Page(s):198 - 202
| | PDF (392 KB)

The design of a planar diode submillimeter-wave mixer is presented in which scale models play an important role. By constructing larger scale models of mixer components, the design process is simplified by making component changes easier and by making the measurements easier with the aid of a network analyzer. Separate scale model measurements were used to find optimum diode geometry, a wide bandw... View full abstract»

• ### Measurement and evaluation of two-phase flow parameters

Publication Year: 1992, Page(s):298 - 303
Cited by:  Papers (9)
| | PDF (492 KB)

The fuzzy set theory and the flow-imaging technique were applied to measure the flow pattern and void fraction of two-phase flow. The flow pattern was identified online by using fuzzy identification', and an instrument system based on the flow-imaging technique was developed for online flow pattern measurement as well. The void fraction was evaluated online by fuzzy multifactorial evaluation'. T... View full abstract»

• ### A low-cost frequency multiplier for synchronous sampling of periodic signals

Publication Year: 1992, Page(s):203 - 207
Cited by:  Papers (40)
| | PDF (380 KB)

The theory of digital signal processing indicates clearly that the measurements performed on periodic signals by means of digital techniques are affected by an error due to the nonsynchronization between the sampling frequency and the signal frequency. The authors present a low-cost device, based on a frequency multiplier, that allows the generation of sampling pulses synchronized with the signal ... View full abstract»

• ### Testing of commercial optical media

Publication Year: 1992, Page(s):304 - 307
Cited by:  Papers (2)
| | PDF (316 KB)

Data integrity is a primary concern for the users of any data storage medium. For optical data storage, the integrity is directly related to the performance and defect level of the storage media and to how they change as the media age. While media materials and thin-film stack designs are chosen with the goal of achieving high data integrity and reliability, variations in the media manufacturing p... View full abstract»

• ### A 100-Msps sampling analog-to-digital converter chip set

Publication Year: 1992, Page(s):208 - 211
Cited by:  Papers (1)  |  Patents (1)
| | PDF (340 KB)

The authors describe a 100-Msps sampling analog-to-digital converter (ADC) chip set that can be used for a wideband high-resolution digital storage oscilloscope (DSO). A high conversion rate and a 500-MHz sampling bandwidth with an 8-b effective resolution are obtained by a combination of three Si-bipolar chips and two GaAs-diode chips. the ADC uses a pipelined two-step subranging architecture tha... View full abstract»

• ### A semifixed frequency method for evaluating the effective resolution of A/D convertors

Publication Year: 1992, Page(s):212 - 217
Cited by:  Papers (17)
| | PDF (380 KB)

A detailed description of a novel algorithm for the `effective bit' test in noncoherent conditions is presented. This algorithm can offer absolute convergence and has the potential for faster implementation than existing methods. A flow diagram for the algorithm and its implementation using the MATLAB package is included. The metrological reliability of input data is checked. An adopted ZEROIN pro... View full abstract»

• ### Some theorems on Walsh transforms of quantizer differential and integral nonlinearity

Publication Year: 1992, Page(s):218 - 225
Cited by:  Papers (23)
| | PDF (432 KB)

Methods for diagnosing weight coefficient tolerances of quantizers are derived, based on the assumption of known and available data about the quantizer's nonlinearities. Two of the methods are direct and based on the Householder orthogonalization, applied to an overdetermined equation system containing data about differential or integral nonlinearities. The remaining two methods are indirect and a... View full abstract»

## Aims & Scope

Papers are sought that address innovative solutions to the development and use of electrical and electronic instruments and equipment to measure, monitor and/or record physical phenomena for the purpose of advancing measurement science, methods, functionality and applications.

Full Aims & Scope

Editor-in-Chief


text:
# Pain/Muscle gain

## Recommended Posts

Ive completed 4 days hard work about 3 days ago.

I was unable to grip anything between my thumb and index finger the next day, the day after it was even worse! today my one hand is much better but the other is still in pain, however there is a significant increase in muscle mass!

I used to weight train many years ago (roughly 8 years ago) and not only are these muscles in my hands larger but in my arms too, almost as if it was just under 8 years ago! why would this be, and why is pain essential (seemingly) for the gain of muscle mass?

secondly, what causes the pain?

##### Share on other sites

http://www.theministryoffitness.com/mof/library/articles/article12.htm

(Edited to correct typo.)

##### Share on other sites

Excellent! thnx

the other data within that URL also answers a few more questions too, some I didnt think thread worthy. although Ide still like to know how its possible for certain muscle groups to become nearly the same size as they were 8 years ago in just 7 days! it hardly seems credible if I didnt see it with my own eyes, but non the less its true! although it rendered those muscle groups practicaly useless for over 3 days, its like a time warp

any ideas?

##### Share on other sites

You're welcome, YT.

Sorry, I don't know much about muscle growth, but that does sound amazing.

##### Share on other sites

it does when you consider Ive lived a very sedentary lifestyle since I got busted up in a bike crash, and spent most of my time in Lab ever since. I did a few days working my ground and another 2 days building a greenhouse, now my arms look like Arnols Schwatzennegers

dont get me wrong, its not a complaint, but it sure generates a few questions

I was exagerating about the Arnie thing

##### Share on other sites

Your muscles sort of get pulled apart in order to be built up again.

Which is fun.

##### Share on other sites

well, I must confess I didnt feel it at the time but it was very cold outside and my "core temp" felt low as well, but boy did I ever feel it the next few days! it still hasnt gone now 100%.

Im just amazed/ gob smacked at the muscle mass increase though in such a short space of time, I really thought Ide become a lamer, but those 4 days HARD GRAFT! sent me back nearly 8 years to my max muscle diameters in those areas/groups. thats the bit I find astonishing

##### Share on other sites

i think there's a set size for muscles in people. if you drop below that set size, it's extremely easy to get back up, but it's hard to get above that.

also, motivation plays a HUGE role in how fast you gain muscle. sounds like you were working hard throughout those 4 days, even when your muscles were tired. sounds like you were working at about the level you would have 8 years ago. that's a very good way to gain muscle (would have been better if you breaked for a day inbetween though).

##### Share on other sites

hmm is there any way to work out how big your muscles are supposed to be apart from looking at your parents?

I heard when Arnie started to bulk up with his training it took him several years to reach his largest possible size (muscle bulk wise) is this possible to have a "limit" to your muscle mass (within reason)

##### Share on other sites

I would have thought the main limits on muscle mass are (i) the volume of protein you can actually assimilate and (ii) the mass of tissue to which you can safely deliver oxygen and remove waste.

##### Share on other sites

I wouldn't be surprised if some of the short term increase in size is due to inflammation.

##### Share on other sites

I just did a test as the muscle mass has no decreased at all and the pain is all but gone, using a standard set of bathroom scales and doing a squeeze test with both hands I reach exactly 15 stone (about 95 kg) of pressure in grip now, I cant maintain it for more than 3.7 seconds, BUT! the max I could do before the rescent work was about 10 stone, so there deffinatly seems to be an increase!

as for imflamation, I actualy thought the exact same thing

and indeed I had inflamation of my joints (very painfull), but now all seems back to normal and fully functional again with nothing in the way of pain other than a slight ache in the one arm.

personaly, Im quite Impressed by this little Fluke observation, although I would love to understand the entire mechanics behind it

##### Share on other sites

great, im off to build a green house now and build some muscles

##### Share on other sites
Sayonara³ said in post # :

I would have thought the main limits on muscle mass are (i) the volume of protein you can actually assimilate and (ii) the mass of tissue to which you can safely deliver oxygen and remove waste.

maybe, but i think it's also a genetic thing

like how people have set weights. it's easy to gain weight up to that point, or lose weight down to that point, but it's hard to go away from that point.

for example: i always stay at around 140-145lb. i haven't tried to gain weight or lose weight, but my body always tries to make me weigh that much no matter how much i eat... or eat enough no matter how much i do.

also: i never strength trained a day in my life (except when it was required for a week in P.E.), yet i am relatively strong (compared to my friends).

##### Share on other sites

I was tough that when you exercise it tears your muscles, then your body goes and fills in the tears and fixes it better than it was.

##### Share on other sites
iglak said in post # :

maybe, but i think it's also a genetic thing

like how people have set weights. it's easy to gain weight up to that point, or lose weight down to that point, but it's hard to go away from that point.

There's some truth in that, as people generally fall into 3 "body type" groups: endo~, ecto~ and mesomorphs. Iirc this does have something to do with how easily they lay on muscle (and maybe fat).

I'm not sure there is a genetic limit on muscle mass though. Sure, it will be very difficult for an ectomorph to get to Arnie-size, but "effort" isn't really a limiting factor per se as far as the actual growth of a muscle goes.

##### Share on other sites

Interesting! I seem to fit the meso-morph catagory then, I do eat well, but only eat once a day.

the "Often becomes lazy" bit certainly seems to fit quite well too

thnx Sayo

np duder

##### Share on other sites

ahh... ineresting. it seems that i am a mesomorph as well.

so then it's partially genetic... i wonder if the "limit" i was talking about has more to do with muscle memory then. perhaps the brain remembered how YT's muscles were used before, allowing him to use them again in the same way. then they would grow to the size required by that level of work.

as for me, i'm a teenager. i never really did any repetitions so i never really had any muscle memory... ignorance is bliss

<edit>

actually, i just thought of something.

Freshman year of highschool i met a kid that took martial arts and had a side hobby of trying to learn martial arts arial tricks (flips and stuff). i was easily able to learn the basics in about 1/8th of the time it took him. then i tried doing harder stuff. i succesfully landed a medium level trick on my first try, but it wasnt very good. then the guy said i needed to keep doing it so that i could build up muscle memory. after he said that my tricking abilities pretty much went downhill...

HIS muscles build up by slowly improving and building muscle memory one step higher each time.

MY muscles build up by ignoring muscle memory and making that giant leap up to the harder trick.

geez, and that fits into the rest of my life too...

*gazes off into space as he ponders his epiphany...*

##### Share on other sites

actually your muscle get hypertrophized and there is pain in this situation for 3-5 days now you should use iodex or such bam . when muscle work against unbearable load the muscle fiber size increases to compansate the load that cause too pain for few days this is what happens in muscular exersise.

##### Share on other sites

Anytime you apply resistance to a muscle it becomes stronger. Check this site out about self-resistance

##### Share on other sites

The gains in muscle mass have to do with muscle memory. When you muscle mass increases it stretches the connective tissue around it. When you weight train again, your muscles grow faster, because the fascia is already stretched.

The pain is Delayed Onset Muscle Soreness, which is theorized to ahve to due with muscle hyperplasia and eccentric repetitions.

##### Share on other sites

I've noticed the same thing. About a year ago I began doing pushups/situps/pullups every night. I'd do about 25-50 of each in an effort to remove some fat and increase muscle mass. After about two months of this, I noticed nice definition and a definite increase in my muscle size. Sadly, I got lazy and quit doing all that working out. My muscles definitely regressed during that time, so upon quitting smoking two weeks ago I decided to start the pushups/situps/pullups again. After two nights of this work, my muscle size and definition was back to where it was before. It was amazing. I couldn't believe how quickly it all came back. I'm still incredibly sore and can't straighten out my arms without pain, but it's pretty remarkable. I also noticed that after doing a bunch of pushups, my hands wouldn't respond as quickly to nerve impulses. I had a bunch of shakes and couldn't grip anything tightly. I even had trouble carrying a glass of water. I thought that was odd, but I attributed that to muscle strain/tiredness and a major build-up of metabolism byproducts in the muscles.

##### Share on other sites

some will be toxins coming out some of the volume increase will be Oxygen and Blood in the mucles shortly after the excersize, thats why most competition show off muscle men/women will "pump up" before a show.

as for the aches, some will be lactic acid in joints as well.

btw, well done on the non-smoking thingy

## Create an account

Register a new account


text:
×
Get Full Access to A First Course In Differential Equations With Modeling Applications - 10 Edition - Chapter 4.7 - Problem 30e
Get Full Access to A First Course In Differential Equations With Modeling Applications - 10 Edition - Chapter 4.7 - Problem 30e

×

# solve the given initial-value problem. Use a | Ch 4.7 - 30E

ISBN: 9781111827052 44

## Solution for problem 30E Chapter 4.7

A First Course in Differential Equations with Modeling Applications | 10th Edition

• Textbook Solutions
• 2901 Step-by-step solutions solved by professors and subject experts
• Get 24/7 help from StudySoup virtual teaching assistants

A First Course in Differential Equations with Modeling Applications | 10th Edition

4 5 1 317 Reviews
19
1
Problem 30E

solve the given initial-value problem. Use a graphing utility to graph the solution curve.

Step-by-Step Solution:

SolutionStep 1In this problem, we have to find the solution of giving differential equation.

Step 2 of 3

Step 3 of 3

##### ISBN: 9781111827052

This full solution covers the following key subjects: curve, given, graph, graphing, initial. This expansive textbook survival guide covers 109 chapters, and 4053 solutions. A First Course in Differential Equations with Modeling Applications was written by and is associated to the ISBN: 9781111827052. This textbook survival guide was created for the textbook: A First Course in Differential Equations with Modeling Applications, edition: 10. The full step-by-step solution to problem: 30E from chapter: 4.7 was answered by , our top Calculus solution expert on 07/17/17, 09:41AM. The answer to “solve the given initial-value problem. Use a graphing utility to graph the solution curve.” is broken down into a number of easy to follow steps, and 14 words. Since the solution to 30E from 4.7 chapter was answered, more than 267 students have viewed the full step-by-step answer.

## Discover and learn what students are asking

Calculus: Early Transcendental Functions : Exponential and Logarithmic Functions
?In Exercises 7-22, solve for $$x$$. $$4^{x}=64$$

#### Related chapters

Unlock Textbook Solution


text:
# Friday math movie: Triumph of the Hexagon

By Murray Bourne, 24 Aug 2012

This video is by reverendgraham, and he says by way of introduction:

An adventure through time and space on a voyage of the HEXAGON, nature's perfect shape.

Well, a lot of people would regard the circle as nature's perfect shape, but never mind, the video is quite nicely put together.

The music is "Excalibur - O Fortuna" (From Carmina Burana).

Be the first to comment below.

### Comment Preview

HTML: You can use simple tags like <b>, <a href="...">, etc.

To enter math, you can can either:

1. Use simple calculator-like input in the following format (surround your math in backticks, or qq on tablet or phone):
a^2 = sqrt(b^2 + c^2)
(See more on ASCIIMath syntax); or
2. Use simple LaTeX in the following format. Surround your math with $$and$$.
$$\int g dx = \sqrt{\frac{a}{b}}$$
(This is standard simple LaTeX.)

NOTE: You can mix both types of math entry in your comment.

## Subscribe

* indicates required

From Math Blogs


text:
# Synopsis: Flexing an Electron Gas

Bending a stack of metal oxide sheets can alter the electrical resistance of a 2D electron gas that resides within.

Flexoelectricity is a polarization that occurs in insulators when they are bent or stretched in a nonuniform way. The effect is usually small, but recent work has explored using it in certain photovoltaic materials. Now researchers have shown that flexing a metal oxide structure—consisting of a lanthanum aluminate (LAO) film above a strontium titanate (STO) substrate—can alter the resistance of a 2D electron gas trapped between the layers. This flexoelectric effect might be useful in future energy-harvesting devices or mechanical sensors.

Thanks to their compact nature and ultralow resistance, 2D electron gases are attractive for electronic device applications. The electron gas in LAO/STO structures is especially interesting, as it exhibits superconductivity and ferromagnetism (see 25 July 2011 Viewpoint). Previous work has shown that the electronic properties of LAO/STO are sensitive to various inputs, such as light and spin-polarized currents. Flexoelectricity offers a new “handle” for manipulating the LAO/STO system.

Ji-yan Dai from The Hong Kong Polytechnic University and colleagues bent an LAO/STO structure while monitoring the resistance through the electron gas. They discovered that the resistance in the gas increased by as much as 3% when the structure’s edges were bent down, forming a “hill” shape. Flexing the sample in a “valley” shape decreased the resistance by as much as 0.3%. The team showed that these resistance changes were caused by flex-induced polarization in the thin film of LAO. The electric field from this polarization altered the resistance in the electron gas by driving electrons into or out of the LAO/STO interface, depending on the flex orientation.

This research is published in Physical Review Letters.

–Michael Schirber

Michael Schirber is a Corresponding Editor for Physics based in Lyon, France.

More Features »

### Announcements

More Announcements »

## Subject Areas

Condensed Matter Physics

## Previous Synopsis

Quantum Information

## Next Synopsis

Nonlinear Dynamics

## Related Articles

Materials Science

### Viewpoint: Phonon Heat Transport Near the Melting Point

Molecular dynamics simulations can fully describe phonon propagation in aluminum, which could enable accurate predictions of phonon thermal conductivity. Read More »

Condensed Matter Physics

### Viewpoint: The Heat in Antiferromagnetic Switching

New experiments suggest that heat might be responsible for the current-induced voltage signals measured in antiferromagnets, and not a rotation of the material’s spins as previously thought. Read More »

Condensed Matter Physics

### Focus: A Graphene Waveguide For Electrons

A new waveguide that uses a nanotube to guide electrons could lead to novel types of circuitry in quantum computers. Read More »


text:
209 views

Suppose you have an event that has 2-equally likely possibilities i.e, $X = a$ and $X = b$.

Clearly, $E(x)$ $=$ $(a+b)/2$ $=$ $1$

and $E(x^2)$ $=$ $(a^2 + b^2)/2$ $=$ $1$

Simplifying above two equations we get : $a + b = 2$ and $a^2 + b^2 = 2$

Putting $a = 2 - b$ on equation 2 and then solving $(2-b)^2 + b^2 = 2$, we get $a = 1$ and $b = 1$

Now,  E(X100) = (1100 + 1100)/2  = 1

edited by
This question is based on probability distribution, you solved it like some apti question. E(x) is the mean.

E(X100) = E( (X50)2 )

Now, as we go on reducing, we get E(X25). Now, E(X25) = E( X24.X)

We know that E(A.B) = E(A). E(B) for independent events A and B.

So, E( X24.X) = E(X24). E(X) = 1    (I am not sure at this step)

reshown

E( X24.X) = E(X24). E(X) = 1

this is correct when one function is independent of other i.e E(x.y) =E(X).E(Y).(when x,y are independent)

CAN WE SAY X24 is independent of X ?

@Agrasar. I think I got the solution. Going with the definition of expectation,

if s={x1,x2.........,xn} is the sample space then,

E(X2) = $\sum_{r\epsilon s}^{}$ X(r=s)2.P(r)     ..................(1)

E(X) = $\sum_{r\epsilon s}^{}$ X(r=s).P(r)        ..................(2)

Now, (1) and (2) are both equal which implies X(r=s)2=X(r=s)       ..................(3)

Now, lets consider what value we get for E(X3).

E(X3)

= $\sum_{r\epsilon s}^{}$ X(r=s)3.P(r)

= $\sum_{r\epsilon s}^{}$ X(r=s)2. X(r=s) .P(r)

= $\sum_{r\epsilon s}^{}$ X(r=s). X(r=s) .P(r)     .................from (3)

= E(X2)

=1

Similarly, we can go on computing for E(X100).

Thus, I think, E(X100) = 1

it should be 1

E(x100) =E(x50 ) (becoz E(X50^2) =E(X100)

=E(X25)

=E(X12 . X)

=E(X6 . X)

=E(X3.X)

=E(X^4)

=E((x^2)^2)

=E(X^2)

=E(X)

=1
@cse23. They have give that E(X2)=E(X) and not E(X100)=E(X50). You are concluding that both things are same which is not correct I think unless they get proved by some formula.

So, both things are different. I had also done the same but later realised that its not correct.
but 100 =(50)^2 ryt?
@cse23. yes but there is difference between x and x^50


text:
MathOverflow will be down for maintenance for approximately 3 hours, starting Monday evening (06/24/2013) at approximately 9:00 PM Eastern time (UTC-4).

Let $D$ be a co-complete category and $C$ be a small category. For a functor $F:C^{op}\times C \to D$ one defines the co-end $$\int^{c\in C} F(c,c)$$ as the co-equalizer of $$\coprod_{c\to c'}F(c,c'){\longrightarrow\atop\longrightarrow}\coprod_{c\in C}F(c,c).$$ It is the indexed co-limit $\mbox{colim}_W F$ where the weight is the functor $W:C^{op}\times C \to Set$ given by $Hom(-,-)$.


text:
## anonymous one year ago What object in the table has the slowest-moving molecules? Object A, Object B, Object C, or Object D?

1. anonymous

|dw:1433175410522:dw| O=object T=temperature (K)

2. anonymous

and here are the numbers just in case 250, 273, 273, 298

3. Michele_Laino

the speed of a molecule is proportional to the square root of the temperature of the gas to which that molecule belongs to, namely the quadratic mean velocity is given by the subsequent formula: $\Large {v_{RMQ}} = \sqrt {\frac{{3RT}}{M}}$ where R is the gas constant, T is the absolute temperature, and M is the molecular mass of a gas

4. anonymous

yes:) so how can we calculate? :/ what do we plug in?

5. Michele_Laino

we have to pick the gas with the highest temperature

6. Michele_Laino

oops. sorry, we have to pick the object with the littlest temperature

7. anonymous

ohh so the one with the slowest moving molecules would be Object D? :O

8. anonymous

ohh okay so it should be object A? is our solution?

9. Michele_Laino

yes! exactly!

10. anonymous

yay! thank you!

11. Michele_Laino

:)


text:
599

My greatest achievement would be when I was 11 years old and I had a back tumor on my spine.  But this is not about that, this is about after I got back from the hospital and I missed three months of school. So, unfortunately, my teacher told me that I would either have to get all B’s that term or else I would have to go to summer school. So I set a goal to get all B’s that term and I ended up succeeding. Plus, for pushing myself through my recovery and achieving my goal I was awarded the Emily Inglis award.

SHARE
Previous article
Next article


text:
Nov 13 2018
5 min

Shiny 1.2.0: Plot caching

We’re pleased to announce the CRAN release of Shiny v1.2.0! This release features Plot Caching, an important new tool for improving performance and scalability in Shiny apps.

If you’re not familiar with the term “caching”, it just means that when we perform a time-consuming operation, we save (cache) the results so that the next time that operation is requested, we can skip the actual operation and instantly fetch the previously cached results. Shiny’s reactive expressions do some amount of caching for you already, and you can use more explicit techniques to cache the various operations you might do to your data (using the memoise package, or manually saving intermediate data frames to disk as CSV or RDS, for two examples).

Plots, being very common and (potentially) expensive-to-compute outputs, are great candidates for caching, and in theory you can use renderImage to accomplish this. But because Shiny’s renderPlot function contains a lot of complex infrastructure code, it’s actually quite a difficult task. Despite some valiant attempts, all of the examples we’ve seen in the wild have had serious limitations we wanted to overcome.

Shiny v1.2.0 introduces a new function, renderCachedPlot, that makes it much easier to add plot caching to your app.

### Using renderCachedPlot

Let’s take a simple, but expensive, plot output:

output$plot <- renderPlot({ ggplot(diamonds, aes(carat, price, color = !!input$color_by)) +
geom_point()
})

The diamonds dataset has 53,940 rows. On my laptop, this takes about 1580 milliseconds (1.58 seconds). Perhaps that’s fast enough for doing exploratory data analysis, but it’s slower than we’d like for a high traffic Shiny app.

We can tell Shiny to cache this plot in two steps.

1. Change renderPlot to renderCachedPlot.
2. Provide a suitable cacheKeyExpr. This is an expression that Shiny will use to determine which invocations of renderCachedPlot should be considered equivalent to each other. (In this case, two plots with different input$color_by values can’t be considered the “same” plot, so the cacheKeyExpr needs to have input$color_by.)
output$plot <- renderCachedPlot({ ggplot(diamonds, aes(carat, price, color = !!input$color_by)) +
geom_point()
}, cacheKeyExpr = { input$color_by })  With these code changes, the first time a plot with a particular input$color_by is requested, it will take the normal amount of time. But the next time it is requested, it will be almost instant, as the previously rendered plot will be reused.

### Benchmarking the results

To quantify the performance difference between the cached and uncached versions, I turned each into a minimal Shiny app (source). This app simply provides the color_by input using the new varSelectInput control, and then renders the output using either of the two code examples above. Then I used our (still-in-development) Shiny load testing tools to record a test script, and “replay” it against both versions of the app, each running in a single R process.

I tested the renderPlot version of the app with 5 concurrent users, and the renderCachedPlot version with 25, 50, and 100 concurrent users. The difference in performance is as dramatic as we’d expect:

With only five concurrent users, the latency is already pretty bad with the renderPlot version. (Note that this isn’t intended to represent typical performance with Shiny apps in general! We chose a particularly torturous ggplot on purpose.)

On the other hand, the renderCachedPlot version doesn’t break a sweat with 50 concurrent users; and even at 100 concurrent users, the latency is still acceptable.

### When to use plot caching

A Shiny app is a good candidate for plot caching if:

1. The app has plot outputs that are time-consuming to generate,
2. These plots are a significant fraction of the total amount of time the app spends thinking, and
3. Most users are likely to request the same few plots.

Since our test has a pretty expensive plot as its only output, and our load testing tools simulate n users all doing the same exact thing, this these numbers reflect a best-case scenario for plot caching.

Shiny can store your cached plots in memory, on disk, or with another backend like Redis. You also have a number of options for limiting the size of the cache. Be sure to read this article to get the most out of this feature!

In future releases of Shiny, we hope to build on this foundation we’ve laid to dramatically speed up other areas of Shiny apps, like reactive expressions and non-plot outputs. In the meantime, we hope you find plot caching to be a useful addition to your performance toolkit!

## Other changes in Shiny v1.2.0

• Upgrade FontAwesome from 4.7.0 to 5.3.1 and made icon tags browsable, which means they will display in a web browser or RStudio viewer by default (#2186). Note that if your application or library depends on FontAwesome directly using custom CSS, you may need to make some or all of the changes recommended in Upgrade from Version 4. Font Awesome icons can also now be used in static R Markdown documents.

• Address #174: Added datesdisabled and daysofweekdisabled as new parameters to dateInput(). This resolves #174 and exposes the underlying arguments of Bootstrap Datepicker. datesdisabled expects a character vector with values in yyyy/mm/dd format and daysofweekdisabled expects an integer vector with day interger ids (Sunday=0, Saturday=6). The default value for both is NULL, which leaves all days selectable. Thanks, @nathancday! #2147

See the full changelog for Shiny v1.2.0 for other minor improvements and bug fixes we’ve made in this release.

Tags:


text:
Question
A hot rock ejected from a volcano's lava fountain cools from $1100^\circ\textrm{C}$ to $40.0^\circ\textrm{C}$, and its entropy decreases by 950 J/K. How much heat transfer occurs from the rock?

$-8.01 \times 10^5 \textrm{ J}$

Solution Video


text:
# Math Help - Set Theory : Help Needed Plz

1. ## Set Theory : Help Needed Plz

How Can I Prove That :

1 - A Is a Subset Or Equal To P(A)
2- P(A/\B) = P(A) /\ P(B)

WHRE A AND B ARE SETS
/\ IS INTERSECTION
Thanks FOR THE HELP IN ADVANCE

2. You must redo your posting!
Read it again. It is impossible for A to equal P(A).
What is B? What does /\ mean?

3. Originally Posted by Plato
You must redo your posting!
Read it again. It is impossible for A to equal P(A).
What is B? What does /\ mean?
/\ MEANS INTERSECTION

What is B?
Do you mean A is a subset of B or of P(B)?
As it is, the question is not yet clear.

5. Originally Posted by Plato
What is B?
Do you mean A is a subset of B or of P(B)?
As it is, the question is not yet clear.
THANK YOU FOR ANSWERING

I HAVE TO Prove That :

given any set A, then i have to prove that
A Is a Subset Or Equal To P(A)

given any 2 sets A and B , then i have to prove that
2- P(A/\B) = P(A) /\ P(B)

/\ IS INTERSECTION
Thanks FOR THE HELP IN ADVANCE

6. Well #1 is FALSE.
The power set of A, P(A), is the collection of all subsets of A.
Now it is true $A \in P(A)$ because $A \subseteq A$.
But cannot happen that $A \subseteq P(A)$.

#2 is simply a matter of noting that $A \cap B \subseteq A\quad \& \quad A \cap B \subseteq B$.
$X \in P\left( {A \cap B} \right)\quad \Rightarrow \quad X \subseteq A \cap B$


text:
# How do I factorize $3{y}^{2}+8yx-3x^{2}$

I am looking for a method that I can use to factorize the following equation $$3{y}^{2}+8yx-3x^{2}$$

Into this: $$(3y-x)(y+3x)$$

-

Hint: Write $3y^2+8yx-3x^2 = x^2 (3z^2 + 8z -3)$, where $z=y/x$. The second factor is a quadratic polynomial in just one variable, and this you perhaps know how to factorize?

-

For what it's worth:

You might try a simpler problem for warm up; say the same problem, but with $x=2$: $$\tag{1} 3y^2+8\cdot 2 y -3\cdot 2^2$$ To factor this, you could guess and check. The first step would be to guess that the factorization of $(1)$ is $$\tag{2} (3y + a)(y+b);$$ since, if you multiplied this out, you would at least get the correct $3y^2$ term in $(1)$.

Now we need to find $a$ and $b$. To get the correct constant term, $-3\cdot2^2$, in $(1)$, make a guess; say, $a=\mp3$ and $b=\pm2^2$. Note here $a$ and $b$ have to have different signs and that we are simply making a guess with $ab=-3\cdot2^2$.

But, would this guess give the right factorization? It would be if the "cross terms" resulting from multiplying $(2)$ out combined to give you $8\cdot2y=16y$. Let's check if that's possible. The cross terms of the product $(2)$ are shown in maroon and green below:

$$(\overbrace{\color{maroon}{3y} \pm \underbrace{\color{darkgreen}3)(\color{darkgreen}y}_{\color{darkgreen}{\mp3y}}\pm\color{maroon}{2^2}}^{\color{maroon}{\pm3y\cdot2^2\ =\ \pm 12y}})$$ But we can't get $16y$ from $\color{maroon}{\pm12y}\mp\color{darkgreen}{3y}$. You'd either have $12y-3y=9y$ or $-12y+3y=-9y$. So our guess isn't correct.

That's ok; try a different guess: say $a=\mp2$, $b=\pm3\cdot2$.

Then you'd have $$(\overbrace{\color{maroon}{3y} \pm \underbrace{\color{darkgreen}2)(\color{darkgreen}y}_{\color{darkgreen}{\mp2y}}\pm\color{maroon}{3\cdot2}}^{\color{maroon}{\pm3y\cdot3\cdot2\ =\ \pm 18y}})$$ Here, we can combine the cross terms to get $16y$, yay! Indeed $16y=\color{maroon}{+18y}+\color{darkgreen}{(-)2y}$. So we need to take $a=-2$ and $b=+3\cdot2$; and the factorization of $(1)$ is $$(3y-2)(y+3\cdot 2).$$

Now back to your problem: factor $$3y^2+ 8xy-3x^2.$$

From our warm up, where $x=2$, you might suspect that the factorization is $$(3y-x)(y+3\cdot x)$$ (this is why I didn't do all the arithmetic in the preceding; that would have made this supposition harder to see).

Checking this guess will reveal that this is indeed the factorization.

Of coure, you could have just used this guess and check procedure on the original expression.

-

Write $8yx$ as $9yx-yx$ to see the factorisation.

\begin{align} 3y^2+8yx-3x^2&=3y^2+9yx-yx-3x^2\\&=3y(y+3x)-x(y+3x)\\&=(3y-x)(y+3x)\end{align}

-

If you know that all the factors will have integer coefficients there's another, more ad hoc approach you can take; the advantage is that it's sometimes easier to apply 'on-the-fly' (for instance, during a test). Write your product as $(ay+bx)(cy+dx)$; then you know that $a\cdot c = 3$ and $b\cdot d=-3$. Since $a$ and $c$ must be factors of 3, either $a=1,c=3$ or $a=3,c=1$ (there are negative versions of these cases, but the negative signs can be absorbed into the $b$ and $d$ factors); likewise, either $b=1, d=-3$, $b=3, d=-1$, $b=-1, d=3$, or $b=-3, d=1$; it's now just a matter of checking $bc+ad$ for the different assignments to see which one gives the correct central coefficient $8$.

-

Hint $\$ By the AC method $\rm\ 3\:\!(3x^2 - 8yx - 3y^2)\: =\: y^2\:\! (X^2 - 8\:\!X - 9),\:$ for $\rm\: X = 3x/y$

-


text:
JEDNOSTKA NAUKOWA KATEGORII A+

Wydawnictwa / Czasopisma IMPAN / Studia Mathematica / Wszystkie zeszyty

Fréchet algebras, formal power series, and automatic continuity

Tom 187 / 2008

Studia Mathematica 187 (2008), 125-136 MSC: Primary 46J05; Secondary 13F25, 46H40. DOI: 10.4064/sm187-2-2

Streszczenie

We describe all those commutative Fréchet algebras which may be continuously embedded in the algebra $\mathbb{C}[[X]]$ in such a way that they contain the polynomials. It is shown that these algebras (except $\mathbb{C}[[X]]$ itself) always satisfy a certain equicontinuity condition due to Loy. Using this result, some applications to the theory of automatic continuity are given; in particular, the uniqueness of the Fréchet algebra topology for such algebras is established.

Autorzy

• S. R. PatelDepartment of Mathematics and Statistics
Indian Institute of Technology
Kanpur 208 016, Uttar Pradesh, India
e-mail
e-mail

Przeszukaj wydawnictwa IMPAN

Zbyt krótkie zapytanie. Wpisz co najmniej 4 znaki.

Odśwież obrazek


text:
# Homework Help: Limit of (x^n + y^n)^(m/n) n->infinity Proof

1. Jul 9, 2011

### BrianMath

1. The problem statement, all variables and given/known data
Show that
$$\lim_{n\to \infty} (x^n + y^n)^{\frac{m}{n}} = (max\{x,y\})^m\;\;\;\forall x,y > 0$$
where $max\{x,y\}$ outputs the greater of the two.

2. Relevant equations
$$\lim_{x\to x_0} (f(x))^n = (\lim_{x\to x_0} f(x))^n$$

3. The attempt at a solution
It's attached as a pdf file. I think I have the proof right, I would just like to know your opinions on how it is laid out. Is it too wordy, or too informal? I'm still not used to writing proofs in a way to convey information to other people, and I've heard that the only way to pick it up is through practice, so that's what I'm doing.

#### Attached Files:

• ###### xnynmn.pdf
File size:
100.4 KB
Views:
94
Last edited: Jul 9, 2011
2. Jul 9, 2011

well, your proof is correct and convincing, and I don't think that it is too informal, you've just explained every step carefully in details which is good, that doesn't make it wordy or informal. you can avoid going into such details in your exam paper. you have to consider the case when x=0 separately if you want it to be more rigorous. (you're not allowed to factor out x^n if it's zero)

Last edited: Jul 9, 2011
3. Jul 9, 2011

### micromass

You're missing the case where x=y

4. Jul 9, 2011

### BrianMath

Ah, yes, thank you.
I've taken care of that, how is it now?

The way I like to write my proofs is to explain every step in detail so that even someone with a basic knowledge of calculus could understand. I think it's mostly to convince myself that I know what I'm doing, but I also want to make sure when someone grades what I do that I don't get points taken away for not providing enough steps or justification.:tongue2:

#### Attached Files:

• ###### xnynmn.pdf
File size:
126 KB
Views:
124
Last edited: Jul 9, 2011
5. Jul 9, 2011

### micromass

Now it's all fine!

I think your proof was quite beautiful. If they showed it to me, then I would say that it's from a textbook. I like how you explained what you're doing, instead of just writing the calculation...
Keep writing proofs like this!!

6. Jul 9, 2011

### BrianMath

Thank you for the compliment! I only ever see proofs that way in textbooks, so I was worried that I might be writing a bit too textbook-stylish (but now I see that's a good thing). I'm self taught for the most part (since I was 13), so I know when I do computations right, but the kind of ambiguity involved in the art of proof writing always made me wary of whatever I write.

Now I'm confident in my proof-writing abilities.


text:
Self Playing Guitar Machine, Adobong Talong Panlasang Pinoy, Thai Iced Coffee Coconut Milk, La Négation Exercices, Acer Chromebook Spin 311 Review, Sanguine Bond Mtg, Togaf Business Architecture Exam, "/> Self Playing Guitar Machine, Adobong Talong Panlasang Pinoy, Thai Iced Coffee Coconut Milk, La Négation Exercices, Acer Chromebook Spin 311 Review, Sanguine Bond Mtg, Togaf Business Architecture Exam, "/>

# ionic bond h2o

###### by

Capillary action and why we see a meniscus. The key difference between hydrogen bond and ionic bond is that ionic bonding exists between permanent anions and cations, whereas hydrogen bonds exist between partial positive and partial negative charges.. Chemical bonds hold atoms and molecules together. 8 years ago. Two forces determine the extent to which the solution will occur: The first step is that the ionic compound is broken down due to the polarity of the surrounding water molecules with oxygen being electronegative and hydrogen being electropositive. Answer Save. There are "in between" states like polar covalent, where one side of the bond is stronger but not fully ionic. This exchange results in a more stable, noble gas electronic configuration for both atoms involved. LiI + H2O. And this I think is the main reason: hydrogen has fairly high Pauling electronegativity (2.20), rather close to oxygen (3.44), which seems polar covalent overall (and why we get hydrogen bonding with water). Ionic bonding is a type of chemical bond in which valence electrons are lost from one atom and gained by another. Cations and Anions Water as a solvent. In the process, ions are separated from each other. Ionic Bond, Dispersion Forces, Ion-dipole forces, Dipole-dipole forces, Hydrogen bonding. This is a list of the solubility rules for ionic solids in water. Lv 7. Relevance. pisgahchemist. Ionic bonds within a salt crystal may be quite strong. Can't be used more than once. However, if the crystal of salt is dissolved in water, each of the individual ions becomes surrounded by water molecules, which inhibit oppositely charged ions from approaching one another closely enough to form ionic bonds. This is the currently selected item. The second step is that both the positive and negative ions are hydrated by the … Surface tension. Cohesion and adhesion of water. Hydrogen bonding in water. The breaking up of Ionic bonds in Polar solvents such as water occurs in two steps. Favorite Answer. 1 Answer. Hence, the Ionic bond has been broken. Hydrogen bonds in water. Specific heat, heat of vaporization, and density of water. Solubility is a result of an interaction between polar water molecules and the ions that make up a crystal. The $\delta^-$ charge on the oxygen atoms of water are attracted to cations and inversely, repels the $\delta^+$ hydrogen atoms. Ionic Bonds. As ionic solids are added to water, water molecules proceed to surround each ion on the surface of the solid, forming a sphere of hydration. Bonds are important in determining the chemical and physical behaviour of molecules and atoms. An ionic bond is based on attractive electrostatic forces between two ions of opposite charge. Importance of water for life.


text:
# GiveWell grant: Open application

We welcome applications for $250,000 in funding for economic empowerment in sub-Saharan Africa, to be disbursed by 12/31/2009. Interested charities should read the full details of our application process and then submit our first-round application. Why we are making this grant: in 2008 we received$250,000 earmarked specifically for regranting to a top organization working on “economic empowerment” (I.e., raising incomes directly as opposed to focusing on education or health outcomes) in sub-Saharan Africa. This grant was associated with an institutional donor that prefers anonymity.

Our recent work on international aid has concluded that economic empowerment is not a particularly promising area for a donor, and we have found no charities in this area that are as promising as our top charities or that have met either of our criteria for further investigation. More at our current writeup on economic empowerment.

We are committed to honoring our donor’s intentions, and with $250,000 to grant, we feel it is possible that we will get access to information we haven’t been able to get access to before. (This serves as something of a “reality check” on the approach we used in our recent research report, which used the information available on charities’ websites as a key indicator of how promising they were – details here.) Thus, we are conducting an application process for this area and this funding in particular. What we hope to accomplish with this grant: we are • Looking to expand a proven, cost-effective, scalable program rather than to fund an “experimental” proposal with no empirical supporting evidence to date. • Looking to help people in sub-Saharan Africa go from extreme poverty to moderate self-sufficiency. (See our definition of these terms.) • Planning to publicly post as much as possible of the materials, and reasoning, behind our decision so that other donors, particularly individual donors, can benefit from the work we do. Because of the larger-than-usual (for us) grant, we recognize that we may be sent more confidential materials than usual, but have a preference for charities that also want to share as much information as possible. All applicants are being strongly encouraged to (and must be explicit if they don’t want to) share their application materials publicly. • Trying to minimize the application/reporting burden on all but the top contenders. Our first-round application does not ask everything we need to know; it is intended to filter out those charities that cannot or will not provide fairly basic, but in our experience fairly rare, information about (a) income/standard of living of clients; (b) details (particularly financials) of past successes in creating self-sustaining operations. Our framework for economic empowerment explains why these two pieces of information are so vital. • Open to funding new research, rather than a program, if we feel it’s necessary. We will most likely award all of the grant money to one or more charities, and are doing what we can to maximize our odds of finding a strong one. If, however, we feel that we truly cannot have confidence in any applicants, we have permission from our funder to use the funds on a formal study of the effectiveness of an existing program (to be carried out by an external evaluator such as the Poverty Action Lab). Note that if we do go this route, we will still be granting all of the$250,000 to one or more other organizations by 12/31/2009 (i.e., any research project we fund will be carried out by other organizations, not by us).

We make grants and recommendations based on substantive and (whenever possible) shareable information, not based on personal relationships, and so we are casting the net as wide as possible. If you know of any great organizations in this area, please make sure they know about our grant.


text:
# Isomorphic rings

## Homework Statement

Hi guys,

I'm trying to show that $$\mathbb{F}_5[x]/(x^2+2)$$ and $$\mathbb{F}_5[x]/(x^2+3)$$ are isomorphic as rings.

## The Attempt at a Solution

As I understand it, I have to find the homomorphism $$\phi:R\to S$$ which is linear and that $$\phi(1)=1$$.

I'm just struggling to find what I need to send $$x$$ to in order to get this work.

$$x^2 + 2 \equiv 0$$
$$(2x)^2 + 2 \equiv 0$$
$$4x^2 + 2 \equiv 0$$
$$4(x^2 + 3) \equiv 0$$
$$x^2 + 3 \equiv 0$$