Can i just replace the code with new V2? Compute the cross-entropy loss: y_cross = y_true * tf.log(y_hat_softmax), Sum over different class for an instance: -tf.reduce_sum(y_cross, reduction_indices=[1]). logits and labels must have the same shape, e.g. Each row is a gradient of one output element s_i with respect to each of its input elements x_j. By clicking Accept all cookies, you agree Stack Exchange can store cookies on your device and disclose information in accordance with our Cookie Policy. Does tensorflow has function to compute the cross entropy according to this formula also? The main purpose of the softmax function is to grab a vector of arbitrary real numbers and turn it into probabilities: (Image by author) The exponential function in the formula above ensures that the obtained values are non-negative. Cross-entropy loss function for the softmax function To derive the loss function for the softmax function we start out from the likelihood function that a given set of parameters of the model can result in prediction of the correct class of each input sample, as in the derivation for the logistic loss function. TensorFlow Sigmoid Cross Entropy with Logits for 1D data. I have noticed that tf.nn.softmax_cross_entropy_with_logits_v2(labels, logits) mainly performs 3 operations: Apply softmax to the logits (y_hat) in order to normalize them: y_hat_softmax = softmax(y_hat). See tf.nn.softmax_cross_entropy_with_logits_v2. Again we use the division rule, but in this case the derivative of the numerator, e^{x_i} with respect to x_j is zero, because j \neq i means the numerator is constant with respect to x_j. All we need is the division rule from calculus. Were only using it for its analytic simplicity to work out the backpropogating error. How to control Windows 10 via Linux terminal? THIS FUNCTION IS DEPRECATED. Taking the log of them will lead those probabilities to be negative values. The cross-entropy loss function is an optimization function that is used for training classification models which classify the data by predicting the probability (value between 0 and 1) of whether the data belong to one class or another. This will also protect against underflow because the denominator will contain a sum of non-negative terms, one of which is e^{x_\text{max} - x_\text{max}} = 1. All rights reserved.Licensed under the Creative Commons Attribution License 3.0.Code samples licensed under the Apache 2.0 License. Not the answer you're looking for? By clicking Post Your Answer, you agree to our terms of service, privacy policy and cookie policy. We shouldnt implement batch cross-entropy this way in a computer. We have discussed SVM loss function, in this post, we are going through another one of the most commonly used loss function, Softmax function. First compute the diagonal entry of row i. Because rows are independently mapped, the Jacobian of row i of \mathbf S with respect to row j \neq i of \mathbf X is a zero matrix. Since softmax is a vector-to-vector transformation, its derivative is a Jacobian matrix. One of the answers you refer to mentions it too: This formulation is often used for a network with one output predicting two classes (usually positive class membership for 1 and negative for 0 output). MLP(ReLu) stops learning after few iterations. Note: I am not an expert on backprop, but now having read a bit, I think the following caveat is appropriate. This is nice because symmetric matrices have great numeric and analytic properties. Note that to avoid confusion, it is required to pass only named arguments to this function. . So each column of \mathbf S^\top is \mathbf s_i. This means that the input to our softmax layer is a row vector with a column for each class. Instead of comparing each element in \(f(x_i;W)\) and return the max value between obtained score and 0, in softmax function, you take the exponential value of the correct class score, \(f_{y_i}\) and then sum up all the exponential value of the scores for each class, which is \(f_j\), the \(j\)-th element of the score vector \(Wx_i\) for image \(x_i\). Softmax is invariant to additively scaling \mathbf x by a constant c. In other words, softmax only cares about the relative differences in the elements of \mathbf x. Can a black pudding corrode a leather tunic? s o f t m a x ( a) = [ a 1 a 2 a N] [ S 1 S 2 S N] And the actual per-element formula is: s o f t m a x j = e a j k = 1 N e a k As its name suggests, softmax function is a soft version of max function. Now, we only care about entries where the row index equals the column index. Tags: It computes softmax cross entropy between logits and labels. Multilabel classification converges to all zeroes. NOTE: While the classes are mutually exclusive, their probabilities need not be. The code borrowed from here demonstrates this perfectly. Softmax is essentially a vector function. Instead well write \mathbf s(\mathbf x) as \mathbf s and s(\mathbf x)_i as s_i, understanding that \mathbf s and s_i are each a function of the entire vector \mathbf x. For example, each CIFAR-10 image is labeled with one and only one label: an image can be a dog or a truck, but not both. To be more specific, the equation above would hold not just for one-hot \mathbf y, but for any \mathbf y specifying a distribution over classes. Your formula is correct, but it works only for binary classification. Unix to verify file has no content and empty lines, BASH: can grep on command line, but not in script, Safari on iPad occasionally doesn't recognize ASP.NET postback links, anchor tag not working in safari (ios) for iPhone/iPod Touch/iPad, Adding members to local groups by SID in multiple languages, How to set the javamail path and classpath in windows-64bit "Home Premium", How to show BottomNavigation CoordinatorLayout in Android, undo git pull of wrong branch onto master, About tf.nn.softmax_cross_entropy_with_logits_v2. ''', """ Why doesn't this unzip all my files in a given directory? If they are not, the computation of the gradient will be incorrect. logits and labels must have the same shape, e.g. def softmax (x): return x.exp () / (x.exp ().sum (-1)).unsqueeze (-1) is used to define the softmax value. The sigmoid cross entropy between logits_1 and logits_2 is: sigmoid_loss = tf.nn.sigmoid_cross_entropy_with_logits(labels = logits_2, logits = logits_1) loss= tf.reduce_mean(sigmoid_loss) The result value is: This criterion computes the cross entropy loss between input and target. We've just seen how the softmax function is used as part of a machine learning network, and how to compute its derivative using the multivariate chain rule. When reading papers or books on neural nets, it is not uncommon for derivatives to be written using a mix of the standard summation/index notation, matrix notation, and multi-index notation (include a hybrid of the last two for tensor-tensor derivatives). It's like comparing apples to oranges. A multiway shootout if you will. The gradient of a dot product, being a linear operation, is just the vector \mathbf y. where we used equation (69) of the matrix cookbook for the derivative of the dot product. In case, the predicted probability of class is way different than the actual class label (0 or 1), the value . Weighted . It takes a integer that indicates the target class of an instance, and the logits, as the inputs, and outputs the cross entropy of the instance. The difference between these two formulas (binary cross-entropy vs multinomial cross-entropy) and when each one is applicable is well-described in this question. The demo code in tensorflow classifies 3 classes. This property of softmax function which generates a probability distribution makes it suitable for probabilistic interpretation in classification tasks. That is, compute the derivative of the ith output, s_i, with respect to its jth input, x_j, where j \neq i. How does DNS work when it comes to addresses after slash? Furthermore, the score function \(f(x_i;W)\) stays the same as SVM describes before. That means our grand Jacobian of \mathbf S with respect to \mathbf X is a diagonal m \times m matrix of n \times n matrices, most of which are zero matrices: Let each row of \mathbf Y be a one-hot label for an example: Then we compute the mean cross-entropy by averaging the cross-entropy of every matching pair of rows of \mathbf Y and \mathbf S. That is, we average over examples, the cross-entropy of each example: The above simplification works because each row of \mathbf S is \mathbf s_i. Instructions for updating: Future major versions of TensorFlow will allow gradients to flow into the labels input on backprop by default. The answer to your second question is yes, there is such a function called tf.nn.sigmoid_cross_entropy_with_logits. From now on, to keep things clear, we wont write dependence on \mathbf x. You can see in the original code that TensorFlow sometimes tries to compute cross entropy from probabilities (when from_logits=False). That is, compute the derivative of the ith output, s_i, with respect to its ith input, x_i. X = torch.randn (batch_size, n_classes) is used to get the values. Be careful in the official documentation: WARNING: This op expects unscaled logits, since it performs a softmax on logits internally for efficiency. Well show how to compute these entries for an arbitrary row i of the Jacobian. Our work thus far considered a single example. Browse other questions tagged, Where developers & technologists share private knowledge with coworkers, Reach developers & technologists worldwide. To interpret the cross-entropy loss for a specific image, it is the negative log of the probability for the correct class that are computed in the softmax function. Notice that we can express this matrix as. If provided, the optional argument weight should be a 1D Tensor assigning weight to each of the classes. What are some tips to improve this product photo? Hence \mathbf x, our input to the softmax layer, was a row vector. torch.nn.functional.cross_entropy(input, target, weight=None, size_average=None, ignore_index=- 100, reduce=None, reduction='mean', label_smoothing=0.0) [source] This criterion computes the cross entropy loss between input and target. To calculate a cross entropy loss that allows backpropagation into both logits and labels, see tf.nn.softmax_cross_entropy_with_logits_v2. Alternatively, if we feed forward a batch of m examples, then \mathbf X contains a row vector for each example in the minibatch. What is the difference between softmax and softmax_cross_entropy_with_logits? A matrix-calculus approach to deriving the sensitivity of cross-entropy cost to the weighted input to a softmax output layer. So the sensitivity of cost to the weighted input to our softmax layer is just the difference of our softmax matrix and our matrix of one-hot labels, where every element is divided by the number of examples in the batch. See the guide: Neural Network > Classification, Computes softmax cross entropy between logits and labels. Backpropagation will happen only into logits. In that case i may only have one value - you can lose the sum over i. Cross-entropy can be calculated using the probabilities of the events from P and Q, as follows: H (P, Q) = - sum x in X P (x) * log (Q (x)) Where P (x) is the probability of the event x in P, Q (x) is the probability of event x in Q and log is the base-2 logarithm, meaning that the results are in bits. To do so, you can substract the maximum value among the array from the entire array, which is demonstrated below: Again, the original input is \([100,400,800]\). Thats because cross-entropy sums the dot products of matching rows of \mathbf Y and \mathbf S. We can sum over matching dot products by using a trace. We take the average of this cross-entropy across all training examples using tf.reduce_mean method. For example, the exponential value of a big value such as 1000 almost goes to infinity, which cause the program returns nan. The answer to your second question is yes, there is such a function called tf.nn.sigmoid_cross_entropy_with_logits. 2018 The TensorFlow Authors. [batch_size, num_classes] and the same dtype (either float16, float32, or float64). This is nice! Making statements based on opinion; back them up with references or personal experience. The shape of output of a softmax is the same as the input - it just normalizes the values. Softmax is a vector-to-vector transformation that turns a row vector, The transformation is easiest to describe element-wise. a = tf.constant (np.array ( [ [.1, .3, .5, .9]])) print s.run (tf.nn.softmax (a)) One of the answers you refer to mentions it too: This formulation is often used for a network with one output predicting two classes (usually positive class membership for 1 and negative for 0 output). What do you call an episode that is not closely related to the main plot? Defined in tensorflow/python/ops/nn_ops.py. We saw that \mathbf s is a distribution. But if you use the softmax and the cross entropy loss, that . The motive of the cross - entropy is to measure the distance from the true values and also used to take the output probabilities. Next creating a function names "sig" for hypothesis function/sigmoid function. where the second term is the n \times n outer product, because we defined \mathbf s as a row vector. This notebook breaks down how `cross_entropy` function is implemented in pytorch, and how it is related to softmax, log_softmax, and NLL (negative log-likelihood). input f is a numpy array Softmax and cross-entropy loss. When the Littlewood-Richardson rule gives only irreducibles? The derivative of softmax is always phrased in terms of softmax. Handling unprepared students as a Teaching Assistant, Return Variable Number Of Attributes From XML As Comma Separated Values. At the same time, we want the loss for the correct class to be 0. Expanding and simplifying, we get. Backpropagation will happen only into logits. Compared to other classes, the probability of the correct class is supposed to be close to 1 for a better classification. This procedure is always true for any element-wise operations. Since \log \mathbf S is an element-wise operation mapping a matrix to a matrix, its Jacobian is a matrix of element-wise derivatives which we chain rule by a Hadamard product, rather than by a dot product. The Softmax Function Softmax function takes an N-dimensional vector of real numbers and transforms it into a vector of real number in range (0,1) which add upto 1. p i = e a i k = 1 N e k a As the name suggests, softmax function is a "soft" version of max function. A 1-D Tensor of length batch_size of the same type as logits with the softmax cross entropy loss. Softmax loss function --> cross-entropy loss function --> total loss function Softmax Cross Entropy Using Numpy Using the softmax cross-entropy function, we would measure the difference between the predictions, i.e., the network's outputs. TensorFlow: Implementing a class-wise weighted cross entropy loss? Find centralized, trusted content and collaborate around the technologies you use most. To correlate with the probability distribution and the loss function, we can apply log function as our loss function because log(1)=0, the plot of log function is shown below: Here, considered the other probability of incorrect classes, they are all between 0 and 1. logits and labels must have the same shape, e.g. Cross entropy loss PyTorch softmax is defined as a task that changes the K real values between 0 and 1. That is \(f(x_i;W)=Wx_i\). loss=nl (pred, target) is used to calculate the loss. See the above mentioned question. In order to prevent this kind of numerical typos, we could normalize the input and avoid of having big values. Now, we multiply the inputs with the weight matrix, and add biases. Softmax is nice because it turns \mathbf x into a probability distribution. All that is required is that each row of labels is a valid probability distribution. So the matrix product \mathbf Y \log \mathbf S^\top dots rows of \mathbf Y with columns of (\log \mathbf S^\top), which is exactly what we want for cross-entropy. softmax function, Deep Learning, CNN Architectures, Inception Networks, Deep Learning, CNN Architectures, ResNet, Residual Blocks, ''' This version is most similar to the math formula, but not numerically stable. # Step 1: compute score vector for each class, # Step 2: normalize score vector, letting the maximum value to 0, #compute the sum of exp of all scores for all classes. One approach is to flatten everything, do a vector-matrix product as before, and then reshape everything, but this is not elegant or intuitive. The next thing we want to consider is how to correlate the computed probability distribution with the loss function. Cross-entropy measures the difference between two probability distributions. since the softmax function is defined as follow: It can be interpreted as the probability of the correct class \(y_i\) given the image \(x_i\), and we want it to be close to 1, meaning we classify this image to its correct class. The vector-to-vector logarithm will have a Jacobian, but since its applied element-wise, the Jacobian will be diagonal, holding each elementwise derivative. Light bulb as limit, to what is current limited to? It's like comparing apples to oranges. How to understand "round up" in this context? To avoid that, we need to add a minus sign when we take log because the minimum loss is 0 and cannot be negative. Is this meat that I was told was brisket in Barcelona the same as U.S. brisket? The entries of the Jacobian take two forms, one for the main diagonal entry, and one for every off-diagonal entry. Since mean cross-entropy maps a matrix to a scalar, its Jacobian with respect to \mathbf S will be a matrix. Removing repeating rows and columns from 2d array. Stack Overflow for Teams is moving to its own domain! The demo code in tensorflow classifies 3 classes. We can write this cost function as. Measures the probability error in discrete classification tasks in which the classes are mutually exclusive (each entry is in exactly one class). Is it possible to make a high-side PNP switch circuit active-low with less than 3 BJTs? Asking for help, clarification, or responding to other answers. I have noticed that tf.nn.softmax_cross_entropy_with_logits_v2(labels, logits) mainly performs 3 operations: Apply softmax to the logits (y_hat) in order to normalize them: y_hat_softmax = softmax(y_hat). Due to the normalization term in the denominator the obtained values sum to 1. The out can be interpreted as a probabilistic output (summing up to 1). Consequences resulting from Yitang Zhang's latest claimed results on Landau-Siegel zeros. The Jacobian has a row for each output element s_i, and a column for each input element x_j. See CrossEntropyLoss for details. Do not call this op with the output of softmax, as it will produce incorrect results. This vector-to-scalar cost function is actually made up of two steps: (1) a vector-to-vector element-wise \log and (2) a vector-to-scalar dot product. Can i just replace the code with new V2? Compute the cross-entropy loss: y_cross = y_true * tf.log(y_hat_softmax), Sum over different class for an instance: -tf.reduce_sum(y_cross, reduction_indices=[1]). Most likely, you'll see something like this: The softmax and the cross entropy loss fit together like bread and butter. """. When the migration is complete, you will access your Teams at stackoverflowteams.com, and they will no longer appear in the left sidebar on stackoverflow.com. See the above mentioned question. It takes n inputs and produces and n outputs. To understand behavior of formula and algorithms it is important to understand the range of values it can take. Due to numerical instabilities clip_by_value becomes then necessary. We are able to do this because of the fact that \mathbf J_{\mathbf X}(\mathbf S) is diagonal, which breaks the matrix-tensor product into an element-wise dot product of gradients and Jacobians. However, the one hot labels includes either 0 or 1, thus the cross entropy for such binary case is formulated as follows shown in here and here: I write code for this formula in the next cell, the result of which is different from above. Weighted cross entropy. In the following code, we will import some libraries from which we can measure the cross-entropy loss softmax. Where the third step followed by the fact that J_{\mathbf X}(\mathbf S) is diagonal. Instead of selecting one maximal value such as SVM, softmax function breaks the whole (sum to 1) into different elements with probability, maximal element getting the largest portion of the distribution while other smaller elements getting relatively small value of it as well. It measures the information gained about our softmax distribution when we sample from our one-hot distribution. torch.nn.functional.cross_entropy takes logits as inputs (performs log_softmax internally) torch.nn.functional.nll_loss is like cross_entropy but takes log-probabilities (log-softmax) values as inputs; And here a quick demonstration: Note the main reason why PyTorch merges the log_softmax with the cross-entropy loss calculation in torch.nn . It's like comparing apples to oranges. However, the one hot labels includes either 0 or 1, thus the cross entropy for such binary case is formulated as follows shown in here and here: I write code for this formula in the next cell, the result of which is different from above. loss function, Note: this formulation is computationally wasteful. Here is the Syntax of tf.nn.softmax_cross_entropy_with_logits () in Python TensorFlow. [batch_size, num_classes] and the same dtype (either float16, float32, or float64). rev2022.11.7.43014. https://www.tensorflow.org/api_docs/python/tf/nn/softmax_cross_entropy_with_logits, https://www.tensorflow.org/api_docs/python/tf/nn/softmax_cross_entropy_with_logits. That means we can protect softmax from overflow by subtracting the maximum element of \mathbf x from every element of \mathbf x. Now since \mathbf y and \mathbf s are each of length m \cdot n, we can reshape this formulation back into matrices, understanding that in both cases the division is element-wise: We apply the chain rule just as before. # Initialize the loss and gradient to zero. What is the difference between a sigmoid followed by the cross entropy and sigmoid_cross_entropy_with_logits in TensorFlow? It seems that y should not be passed to a softmax function. While we're at it, it's worth to take a look at a loss function that's commonly used along with softmax for training a network: cross-entropy. We use row vectors and row gradients, since typical neural network formulations let columns correspond to features, and rows correspond to examples.This means that the input to our softmax layer is a row vector with a column for each class. Tensor Flow. Why am I getting some extra, weird characters when making a file from grep output? This is particularly useful when you have an unbalanced training set. cross entropy, Now, we have computed the score vectors for each image \(x_i\) and have implemented the softmax function to somehow transform the numerical scores to probability distribution. The Softmax regression is a form of logistic regression that normalizes an input value into a vector of values that follows a probability distribution whose total sums up to 1. Since our \mathbf y is given and fixed, cross-entropy is a vector-to-scalar function of only our softmax distribution. We can see this by concatenating the rows of \mathbf S. such that \mathbf s is a row vector of length m \cdot n. Then \log is an element-wise vector-to-vector transformation again. Parameters Do not call this op with the output of softmax, as it will produce incorrect results. (deprecated). Site design / logo 2022 Stack Exchange Inc; user contributions licensed under CC BY-SA. To calculate a cross entropy loss that allows backpropagation into both logits and labels, see tf.nn.softmax_cross_entropy_with_logits_v2. My question is which one is better or right? Multilabel image classification with sparse labels in TensorFlow? We owe this entirely to the fact that softmax is a row-to-row transformation, such that its Jacobian tensor is diagonal. The second term is the Jacobian of softmax activation to softmax input. Does tensorflow has function to compute the cross entropy according to this formula also? [batch_size, num_classes] and the same dtype (either float16, float32, or float64). Remember the takeaway is: the essential goal of softmax is to turn numbers . The last line follows from the fact that \mathbf y was one-hot and applied to a matrix whose rows are identically our softmax distribution. In tensorflow, you can use the sparse_sof tmax_cross_entropy_with_logits () function to do the tasks of Softmax and computing the cross entropy. System information. For example, each CIFAR-10 image is labeled with one and only one label: an image can be a dog or a truck, but not both. Remember that were using row gradients - so this is a row vector times a matrix, resulting in a row vector. By the chain rule, the sensitivity of cost H to the input to the softmax layer, \mathbf x, is given by a gradient-Jacobian product, each of which weve already computed: The first term is the gradient of cross-entropy to softmax activation. This formula comes from information theory. The softmax transfer function is typically used to compute the estimated probability distribution in classification tasks involving multiple classes. If you apply a softmax on your output, the loss calculation would use: loss = F.nll_loss (F.log_softmax (F.softmax (logits)), target) which is wrong based on the formula for the cross entropy loss due to the additional F . l = i y i l o g ( a i) Where l is the actual loss. That means it will have a gradient with respect to our softmax distribution. Connect and share knowledge within a single location that is structured and easy to search. Softmax function is defined as below: It can be interpreted as the probability assigned to the correct label \(y_i\) given the training image, \(x_i\) parameterized by \(W\). The mapping function \(f:f(x_i;W)=Wx_i\) stays unchanged, but we now interpret these scores as the unnormalized log probabilities for each class and we could replace the hinge loss/SVM loss with a cross-entropy loss that has the form: where \(f_{y_i}\) is the probability for correct class score and \(f_j\) is the \(j\)-th element of the score vector for each image. The correct class is also a distribution if we encode it as a one-hot vector: where the 1 appears at the index of the correct class of this single example. Instead, we dot rows of \mathbf J_{\mathbf S}(H), each a gradient of a row-wise cross-entropy, against diagonal elements of \mathbf J_{\mathbf X}(\mathbf S), each a Jacobian matrix of a row-wise softmax. Multiplying a matrix against a tensor is difficult. Itll drive our softmax distribution toward the one-hot distribution. In my case where logits and labels have shape [2,3,4], I currently use following function - def softmax_and_cross_entropy(logits, labels): return -(labels * nn.LogSoftmax(dim=2)(logits)).sum(dim=2) I would like to know if there is a better way to go about it so that the function could be in a more pytorch style and backward could also get faster. Logits values are essentially. Tensorflow: Convolutional Neural Networks in Tensorflow(without Keras), Saving and Loading Models (Coding TensorFlow), Upgrade your existing code for TensorFlow 2.0 (Coding TensorFlow), Use TensorFlow to classify clothing images (Coding TensorFlow), TensorFlow 2.0 Complete Course - Python Neural Networks for Beginners Tutorial, What is the difference of this V2 to the previous one? To learn more, see our tips on writing great answers. See the above mentioned question. It will be removed in a future version. If you like my content, please consider buying me a coffee. The softmax "squishes" the inputs so that sum (input) = 1; it's a way of normalizing. The only difference is that our gradient-Jacobian product is now a matrix-tensor product. The ith output s(\mathbf x)_i is a function of the entire input \mathbf x, and is given by. So it has an m \cdot n \times m \cdot n diagonal Jacobian matrix. After normalization, the vector becomes \([-700,-400,0]\), which avoids the occurrence of nan. Does Ape Framework have contract verification workflow? A very good link I stumbled upon is this one: The difference between these two formulas (binary cross-entropy vs multinomial cross-entropy) and when each one is applicable is well-described in this question. and the Jacobian of row i of \mathbf S with respect to row i of \mathbf X is our familiar matrix from before. backpropogation, matrix calculus, softmax, cross-entropy, neural networks, deep learning. As its name suggests, softmax function is a "soft" version of max function. Softmax transfer function: \begin{equation} \hat{y}_i = \frac{e^{z_i}}{\sum_k e^{z_k}} \end{equation} where is the -th pre-activation unit. In this post, I will always assume that tf.keras.layers.Sigmoid() is not applied (or only during prediction). Measures the probability error in discrete classification tasks in which the classes are mutually exclusive (each entry is in exactly one class). Cross-entropy loss is the measure that you use in order to see how well the score this function give is "good" compared to what you expect. The form of the off-diagonals tells us that the Jacobian of softmax is a symmetric matrix. Can an adult sue someone who violated them as a child? 503), Mobile app infrastructure being decommissioned. Which finite projective planes can have a symmetric incidence matrix? What is the difference of this V2 to the previous one? We expand it below. There are many types of loss functions as mentioned before. Code First, importing a Numpy library and plotting a graph, we are importing a matplotlib library. We compute the softmax and cross-entropy using tf.nn.softmax_cross_entropy_with_logits (it's one operation in TensorFlow, because it's very common, and it can be optimized).
Words With Friends 2 Word Game, South Korea Vs Paraguay Results, Fluke 196c Scopemeter Manual, Compound Growth And Decay Worksheet, Student Union Society,