Integration by parts is not just a way of evaluating integrals explicitly, like $\int x \sin(x) dx$ or the occasional trick with $\int \ln(x) dx$. Let me introduce the newest Theorem, in the highest sense of the word, to your analysis toolbox.
Let $f \in C^n([a, x])$ for some $n \in \mathbb{N}$. Then $$f(x) = \sum_{k=0}^{n-1} \frac{f^{(k)}(a)}{k!} (x-a)^k + R_n(x)$$ where the remainder term is $$ R_n(x) = \int_a^x \frac{(x-t)^{n-1}}{(n-1)!} f^{(n)}(t) dt$$
Assume inductively for $J < n$ that $$f(x) = \sum_{k=0}^{J-1} \frac{f^{(k)}(a)}{k!}(x-a)^k + \int_a^x \frac{(x-t)^{J-1}}{(J-1)!} f^{(J)}(t) dt$$ Applying GIBP to the integral, we have $$\begin{align*} f(x) - \sum_{k=0}^{J-1} \frac{f^{(k)}(a)}{k!}(x-a)^k &= (-1)^{J-1}\int_a^x f^{(J)}(t) d \left( \frac{(t-x)^J}{J!} \right) \\ &= (-1)^{J-1} f^{(J)}(t) \left. \frac{(t-x)^J}{J!} \right|_a^x - (-1)^{J-1} \int_a^x f^{(J+1)}(t) \frac{(t-x)^J}{J!} dt \\ &= \frac{f^{(J)}(t)}{J!} (x-a)^J + \int_a^x \frac{(x-t)^J}{J!} f^{(J+1)}(t) dt\\ &= \frac{f^{(J)}(t)}{J!} (x-a)^J + R_{J+1}(x) \end{align*} $$ So collapsing into the summation yields, as desired, $$ f(x) = \sum_{k=0}^{J} \frac{f^{(k)}(a)}{k!}(x-a)^k + R_{J+1}(x)$$
In particular, showing that a special case of the Riemann-Lebesgue theorem$\text{\textemdash}$very important to Fourier analysis, one of my favorite branches of math$\text{\textemdash}$holds:
Let $f \in C_0(\mathbb{R})$ [continuous functions compactly supported in $\mathbb{R}$]. Show that $$ \lim_{n \to \infty} \int f(x) \sin(nx) dx = 0$$ Hint: Assume first that $f \in C_0^\infty(\mathbb{R})$.
Proof. Assume that $f$ is infinitely differentiable$\text{\textemdash}$that is, we take a function that is epsilon-close to $f$ or less than that at all points, which exists by Stone-Weierstrass Theorem. (Note that we have satisfied the condition of being on a compact interval, say $[-R, R]$, since $f$ is compactly supported.) Then we apply the Goated Integration by Parts to it, and choose $f$ to be the part differentiated: $$ \begin{align*} \int f(x) \sin(nx) dx &= \int_{-R}^{R} f(x) \sin(nx) dx\\ &= \left. -\frac{1}{n} \cos(nx) f'(x) \right|_{-R}^{R}+ \frac{1}{n} \int_{-R}^{R} \cos(nx) f'(x) dx \\ &= \frac{1}{n} \int_{-R}^{R} \cos(nx) f'(x) dx \end{align*} $$ by cosine's evenness. We want this to go to zero with $n$. Though this quantity depends on $n$ inside the integral, cosine's value never exceeds $1$, so the magnitude of this integral is just a number no bigger than $$ \left| \frac{1}{n} \int_{-R}^{R} f'(x) dx \right| \leq \frac{1}{n} \int_{-R}^{R} \left| f'(x) \right| dx $$ And since $f'$ is continuous and thus integrable over $[-R, R]$, its maximum value is some constant $M < \infty$: $$ \frac{1}{n} \int_{-R}^{R} \left| f'(x) \right| dx \leq \frac{1}{n} \cdot 2R \cdot M$$ Thus, $\int f(x) \sin(nx) dx$ tends to $0$ as $n \to \infty$, just as desired!
However, this was not our real function but an impostor procured by the Stone-Weierstrass Theorem, epsilon-close. But intuitively, if we let $f_\epsilon$ denote our uniform approximation function to $f$, it makes sense that our smallness (in a sense) of $f_\epsilon$ would imply the smallness (in the same sense) for $f$ since the two are very close. To show this formally, we use the classic "add zero and apply the triangle inequality" trick: $$ \begin{align*} \left| \int f(x) \sin(nx) dx \right| &= \left| \int (f(x) - f_\epsilon(x)) \sin(nx) dx + \int f_\epsilon(x) \sin(nx) dx \right| \end{align*} $$ by linearity of integrals. By triangle inequality we bound this from above by $$\leq \int \left| f(x) - f_\epsilon(x) \right| dx + \left| \int f_\epsilon(x) \sin(nx) dx \right| $$ All that is left is unpacking these terms to show that they are arbitrarily small. Replacing the second term with our result above, $$ \leq \int \left| f(x) - f_\epsilon(x) \right| dx + \frac{1}{n} 2R M $$ So choosing epsilon to be $1/n$$\text{\textemdash}$and thus choosing different approximation functions$\text{\textemdash}$we can make this quantity as small as we want, bounding our integral from above: $$ \leq \int \left| f(x) - f_\frac{1}{n}(x) \right| dx + \frac{1}{n} 2RM \leq \frac{1}{n}2R + \frac{1}{n}2RM $$ which tends to $0$ as $n \to \infty$ as desired.