Move this out of the common header and include it only where needed. In
a number of cases this requires adding "struct udevice;" to avoid adding
another large header or in other cases replacing / adding missing header
files that had been pulled in, very indirectly. Finally, we have a few
cases where we did not need to include <asm/global_data.h> at all, so
remove that include.
Signed-off-by: Simon Glass <sjg@chromium.org>
Signed-off-by: Tom Rini <trini@konsulko.com>
Force inlining of IOC related functions used in other cache
functions. This is preventive change.
Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Signed-off-by: Alexey Brodkin <abrodkin@synopsys.com>
Since version 3.0 ARC HS supports SL$ (L2 system level cache)
disable. So add support for SL$ disable/enable to code.
Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Signed-off-by: Alexey Brodkin <abrodkin@synopsys.com>
These functions belong in cpu_func.h. Another option would be cache.h
but that code uses driver model and we have not moved these cache
functions to use driver model. Since they are CPU-related it seems
reasonable to put them here.
Move them over.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Tom Rini <trini@konsulko.com>
We don't care much about I$ line length really as there're
no per-line ops on I$ instead we only do full invalidation of it
on occasion of relocation and right before jumping to the OS.
Also as compared to Linux kernel where we don't support different
lengths of I$ and D$ lines in U-Boot we have to deal with such an
exotic configs if the target board is not supposed to run Linux kernel.
Signed-off-by: Alexey Brodkin <abrodkin@synopsys.com>
When U-Boot started using SPDX tags we were among the early adopters and
there weren't a lot of other examples to borrow from. So we picked the
area of the file that usually had a full license text and replaced it
with an appropriate SPDX-License-Identifier: entry. Since then, the
Linux Kernel has adopted SPDX tags and they place it as the very first
line in a file (except where shebangs are used, then it's second line)
and with slightly different comment styles than us.
In part due to community overlap, in part due to better tag visibility
and in part for other minor reasons, switch over to that style.
This commit changes all instances where we have a single declared
license in the tag as both the before and after are identical in tag
contents. There's also a few places where I found we did not have a tag
and have introduced one.
Signed-off-by: Tom Rini <trini@konsulko.com>
Move all checks before cache flush and IOC setup.
Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Signed-off-by: Alexey Brodkin <abrodkin@synopsys.com>
Add missing cache cleanup before cache disable:
* Flush and invalidate L1 D$ before disabling. Flush and invalidate
SLC before L1 D$ disabling (as it will be bypassed for data)
Otherwise we can lose some data when we disable L1 D$ if this data
isn't flushed to next level cache. Or we can get wrong data if L1 D$
has some entries after enable which we modified when the L1 D$ was
disabled.
* Invalidate L1 I$ before disabling. Otherwise we can execute wrong
instructions after L1 I$ enable if we modified any code when
L1 I$ was disabled.
Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Signed-off-by: Alexey Brodkin <abrodkin@synopsys.com>
Add additional cache configuration checks and note about
supported configurations.
It is unlikely to face some configuration in real life but
it's better to be prepared and refuse to work on those.
Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Signed-off-by: Alexey Brodkin <abrodkin@synopsys.com>
Implement specialized function to clenup caches (and therefore
sync instruction and data caches) which can be used for cleanup before linux
launch or to sync caches during U-Boot self-relocation.
Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Signed-off-by: Alexey Brodkin <abrodkin@synopsys.com>
If L1 D$ is disabled SLC is bypassed for data and all
load/store requests are sent directly to main memory.
If L1 I$ is disabled SLC is NOT bypassed for instructions
and all instruction requests are fetched through SLC.
Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Signed-off-by: Alexey Brodkin <abrodkin@synopsys.com>
Implement icache_enabled() and dcache_enabled() as separate functions
which can be used with "inline" attribute. This is a preparation to
make them always_inline.
Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Signed-off-by: Alexey Brodkin <abrodkin@synopsys.com>
Move check for PAE existence into slc_upper_region_init()
instead of its caller as more appropriate place.
Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Signed-off-by: Alexey Brodkin <abrodkin@synopsys.com>
There is a problem with current implementation if we start U-Boot
from ROM, as we use global variables before ther initialization,
so these variables get overwritten when we copy .data section
from ROM.
Instead we move these global variables into our "global data"
structure so that we may really start from ROM.
Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Signed-off-by: Alexey Brodkin <abrodkin@synopsys.com>
There is a problem with current implementation if we start U-Boot
from ROM, as we use global variables before ther initialization,
so these variables get overwritten when we copy .data section
from ROM.
Instead we'll use icache_exists(), dcache_exists(), slc_exists(), pae_exists()
functions which directly check BCRs every time.
In U-Boot case ops are used only during self-relocation and DMA
so we shouldn't be hit by noticeable performance degradation.
Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Signed-off-by: Alexey Brodkin <abrodkin@synopsys.com>
As of today we check SLC status before each call of __slc_rgn_op()
or __slc_entire_op(). So move status check into __slc_rgn_op()
and __slc_entire_op().
As we need to check status before *each* function execution and we
call slc_entire_op() and slc_rgn_op() from different places we add
this check directly into SLC entire/line functions instead of
their callers to avoid code duplication.
Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Signed-off-by: Alexey Brodkin <abrodkin@synopsys.com>
Use is_isa_arcv2() function where it is possible instead of
CONFIG_ISA_ARCV2 define check to make code cleaner at the same time
keeping pretty much the same functionality - code in branches
under "if (is_isa_arcv2())" won't be compiled if CONFIG_ISA_ARCV2
is not defined, still we need a couple of CONFIG_ISA_ARCV2
ifdefs to make compiler happy. That's because code in
!is_isa_x() branch gets compiled and only then gets optimized
away.
Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Signed-off-by: Alexey Brodkin <abrodkin@synopsys.com>
As we are planning to get rid of dozens of ifdef's in cache.c we
would better check D$ status before each entire/line operation
then check CONFIG_SYS_DCACHE_OFF config option.
This makes the code cleaner as well as D$ entire/line functions
remain functional even if we enable or disable D$ in run-time.
As we need to check status before *each* function execution and we
call D$ entire/line functions from different places we add
this check directly into D$ entire/line functions instead of
their callers to avoid code duplication.
Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Signed-off-by: Alexey Brodkin <abrodkin@synopsys.com>
We're starting to use more and more BCRs and having their
definitions in-lined in sources becomes a bit annoying
so we move it all to a separate header.
Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Signed-off-by: Alexey Brodkin <abrodkin@synopsys.com>
Move IOC initialization from cache_init() to a separate function.
This is the preparation for the next patch where we'll switch
to is_isa_arcv2() function usage instead of "CONFIG_ISA_ARCV2"
ifdef.
Also it makes cache_init function a bit cleaner.
Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Signed-off-by: Alexey Brodkin <abrodkin@synopsys.com>
We don't implement separate flush_dcache_all() intentionally as
entire data cache invalidation is dangerous operation even if we flush
data cache right before invalidation.
There is the real example:
We may get stuck in the following code if we store any context (like
BLINK register) on stack in invalidate_dcache_all() function.
BLINK register is the register where return address is automatically saved
when we do function call with instructions like 'bl'.
void flush_dcache_all() {
__dc_entire_op(OP_FLUSH);
// Other code //
}
void invalidate_dcache_all() {
__dc_entire_op(OP_INV);
// Other code //
}
void foo(void) {
flush_dcache_all();
invalidate_dcache_all();
}
Now let's see what really happens during that code execution:
foo()
|->> call flush_dcache_all
[return address is saved to BLINK register]
[push BLINK] (save to stack) ![point 1]
|->> call __dc_entire_op(OP_FLUSH)
[return address is saved to BLINK register]
[flush L1 D$]
return [jump to BLINK]
<<------
[other flush_dcache_all code]
[pop BLINK] (get from stack)
return [jump to BLINK]
<<------
|->> call invalidate_dcache_all
[return address is saved to BLINK register]
[push BLINK] (save to stack) ![point 2]
|->> call __dc_entire_op(OP_FLUSH)
[return address is saved to BLINK register]
[invalidate L1 D$] ![point 3]
// Oops!!!
// We lose return address from invalidate_dcache_all function:
// we save it to stack and invalidate L1 D$ after that!
return [jump to BLINK]
<<------
[other invalidate_dcache_all code]
[pop BLINK] (get from stack)
// we don't have this data in L1 dcache as we invalidated it in [point 3]
// so we get it from next memory level (for example DDR memory)
// but in the memory we have value which we save in [point 1], which
// is return address from flush_dcache_all function (instead of
// address from current invalidate_dcache_all function which we
// saved in [point 2] !)
return [jump to BLINK]
<<------
// As BLINK points to invalidate_dcache_all, we call it again and
// loop forever.
Fortunately we may do flush and invalidation of D$ with a single one
instruction which automatically mitigates a situation described above.
And because invalidate_dcache_all() isn't used in common U-Boot code we
implement "flush and invalidate dcache all" instead.
Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Signed-off-by: Alexey Brodkin <abrodkin@synopsys.com>
As of today __dc_line_op() and __dc_entire_op() support
only separate flush (OP_FLUSH) and invalidate (OP_INV) operations.
Add support of combined flush and invalidate (OP_FLUSH_N_INV)
operation which we planing to use in subsequent patches.
Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Signed-off-by: Alexey Brodkin <abrodkin@synopsys.com>
__cache_line_loop() function was copied from Linux kernel
where per-line instruction cache operations are really used.
In U-Boot we use only entire I$ ops, so we can drop support of
per-line I$ ops from __cache_line_loop() because __cache_line_loop()
is never called with OP_INV_IC parameter.
Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Signed-off-by: Alexey Brodkin <abrodkin@synopsys.com>
Move instruction cache entire operation to a separate function
because we are planing to use it in other places like
sync_icache_dcache_all().
Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Signed-off-by: Alexey Brodkin <abrodkin@synopsys.com>
We'd like to keep IOC HW at the same state as t is right after reset when we
start Linux kernel so there will be no re-configuration of IOC on the go.
The point is U-Boot doesn't benefit a lot from IOC as it doesn't do a
lot of DMA operations especially on multiple cores simultaneously.
At the same time re-configuration of IOC in run-time might become quite
a tricky experience because we need to make sure there're no DMA
trannsactions in flight otherwise unexpected consequencses might affect
us much later and debugging those kinds of issues will be a real
nightmare.
That said let's make our life easier a little bit.
Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Signed-off-by: Alexey Brodkin <abrodkin@synopsys.com>
Previous SLC management implementation is broken. Seems like it was
never sufficiently tested probably because most of the time IOC was used
instead (i.e. no manual cache operations were done).
Now if we disable IOC in U-boot we'll get a lot of errors while using
DMA-enabled peripherals.
This time we fix it by substitution of broken per-line SLC operations
region operations as it is done in the Linux kernel (we took it from
v4.14 which is the latest stable as of today).
Among other things this implementation might be a bit faster because
instead of iteration over each and every cache line we're taking care
about entire region in one go.
Main changes:
* Replaced __slc_line_op (per line operations) by __slc_rgn_op
(region operations).
* Reworked __slc_entire_op to get rid of __after_slc_op and
__before_slc_op functions.
Note flush fix (flush only instead of flush-n-inv when OP_FLUSH is
used, see [1] for more details) is already incorporated here.
* Added SLC invalidation to invalidate_icache_all().
* Added (start >= end) check to invalidate_dcache_range() and
flush_dcache_range() as some buggy drivers pass region start == end.
* Added read-out of MMU BCR so we may know if PAE40 exists in HW and then
act on a particular AUX regs accordingly.
[1] http://lists.infradead.org/pipermail/linux-snps-arc/2018-January/003357.html
Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Signed-off-by: Alexey Brodkin <abrodkin@synopsys.com>
dcache_exists, icache_exists, slc_exists and ioc_exists global
variables in "arch/arc/lib/cache.c" remain uninitialized if
SoC doesn't have corresponding HW.
This happens because we use the next constructions for their
definition and initialization:
-------------------------->>---------------------
int ioc_exists __section(".data");
if (/* condition */)
ioc_exists = 1;
-------------------------->>---------------------
That's quite a non-trivial issue as one may think of it.
The point is we intentionally put those variables in ".data" section
so they might survive relocation (remember we initilaize them very early
before relocation and continue to use after reloaction). While being
non-initialized and not explicitly put in .data section they would end-up
in ".bss" section which by definition is filled with zeroes.
But since we place those variables in .data section we need to care
about their proper initialization ourselves.
Also while at it we change their type to "bool" as more appropriate.
Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Signed-off-by: Alexey Brodkin <abrodkin@synopsys.com>
As per ARC HS databook (see chapter 5.3.3.2) it is required to add
3 NOPs after each write to IC_IVIC which we do from now on.
Signed-off-by: Alexey Brodkin <abrodkin@synopsys.com>
Cc: Eugeniy Paltsev <paltsev@synopsys.com>
We used to use the same memory layout and size for a couple of
boards and thus we just hardcoding IOC aperture start and size.
Now when we're getting more boards with more memory on board we
need to have an ability to set IOC so it matches real DDR layout
and size.
Even though it is not really a must but for simplicity we assume
IOC covers all the DDR we have, that gives us a chance to not
bother where DMA buffers are allocated - any part of DDR is OK.
Signed-off-by: Alexey Brodkin <abrodkin@synopsys.com>
As reported in STAR 9001165532, an SLC control reg read (for checking
busy state) right after SLC invalidate command may incorrectly return
NOT busy causing software to NOT spin-wait while operation is underway.
(and for some reason this only happens if L1 cache is also disabled - as
required by IOC programming model)
Suggested workaround is to do an additional Control Reg read, which
ensures the 2nd read gets the right status.
Same fix made in Linux kernel:
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=c70c473396cbdec1168a6eff60e13029c0916854
Signed-off-by: Alexey Brodkin <abrodkin@synopsys.com>
According to ARC HS databook it is required to flush and disable
caches prior programming IOC registers. Otherwise ongoing coherent
memory operations may not observe the coherency protocols as
expected.
But since in ARC HS v2.1 there's no way to disable SLC (AKA L2 cache)
we're doing our best flushing and invalidating it.
Signed-off-by: Alexey Brodkin <abrodkin@synopsys.com>
invalidate_dcache_all() could be used in different use-cases
and what is especially important most of those cases won't be
related to DMAed data to or from peripherals, i.e. we'll be doing
invalidation of data used purely by CPU cores.
Given that IOC engine only snoops data that goes through DMA
we need to care ourselves about data used only by CPU cores
and so remove dependency on IOC from invalidate_dcache_all()
and always do real invalidation.
Signed-off-by: Alexey Brodkin <abrodkin@synopsys.com>
flush_dcache_all() is used in the very end of U-Boot self relocation
to write back all copied and then patched code and data to their
new location in the very end of available memory space.
Since that has nothing to do with IO (i.e. no external DMA happens
here) IOC won't help here and we need to write back data cache contents
manually.
Signed-off-by: Alexey Brodkin <abrodkin@synopsys.com>
With release of ARC HS38 v2.1 new IO coherency engine could be built-in
ARC core. This hardware module ensures coherency between DMA-ed data
from peripherals and L2 cache.
With L2 and IOC enabled there's no overhead for L2 cache manual
maintenance which results in significantly improved IO bandwidth.
Signed-off-by: Alexey Brodkin <abrodkin@synopsys.com>
ARC core could be configured with different L1 and L2 (AKA SLC) cache
line lengths. At least these values are possible and were really used:
32, 64 or 128 bytes.
Current implementation requires cache line to be selected upon U-Boot
configuration and then it will only work on matching hardware. Indeed
this is quite efficient because cache line length gets hardcoded during
code compilation. But OTOH it makes binary less portable.
With this commit we allow U-Boot to determine real L1 cache line length
early in runtime and use this value later on. This extends portability
of U-Boot binary a lot.
Signed-off-by: Alexey Brodkin <abrodkin@synopsys.com>
[1] Align cache management functions to those in Linux kernel. I.e.:
a) Use the same functions for all cache ops (D$ Inv/Flush)
b) Split cache ops in 3 sub-functions: "before", "lineloop" and
"after". That way we may re-use "before" and "after" functions for
region and full cache ops.
[2] Implement full-functional L2 (SLC) management. Before SLC was
simply disabled early on boot. It's also possible to enable or disable
L2 cache from config utility.
[3] Disable/enable corresponding caches early on boot. So if U-Boot is
configured to use caches they will be used at all times (this is useful
in partucular for speed-up of relocation).
Signed-off-by: Alexey Brodkin <abrodkin@synopsys.com>
ARCv2 cores may have built-in SLC (System Level Cache, AKA L2-cache).
This change adds functions required for controlling SLC:
* slc_enable/disable
* slc_flush/invalidate
For now we just disable SLC to escape DMA coherency issues until either:
* SLC flush/invalidate is supported in DMA APIin U-Boot
* hardware DMA coherency is implemented (that might be board specific
so probably we'll need to have a separate Kconfig option for
controlling SLC explicitly)
Signed-off-by: Alexey Brodkin <abrodkin@synopsys.com>
always
Make both invalidate_icache_all() and invalidate_dcache_all() available
even if U-Boot is configured with CONFIG_SYS_DCACHE_OFF and/or
CONFIG_SYS_ICACHE_OFF.
This is useful because configuration of U-Boot may not match actual
hardware features. Real board may have cache(s) but for some reason we
may want to run U-Boot with cache(s) disabled (for example if some
peripherals work improperly with existing drivers if data cache is
enabled). So board may start with cache(s) enabled (that's the case for
ARC cores with built-in caches) but early in U-Boot we disable cache(s)
and make sure all contents of data cache gets flushed in RAM.
Signed-off-by: Alexey Brodkin <abrodkin@synopsys.com>
This change allows to keep board description clean and minimalistic.
This is especially helpful if one board may house different CPUs with
different features.
It is applicable to both FPGA-based boards or those that have CPUs
mounted on interchnagable daughter-boards.
Signed-off-by: Alexey Brodkin <abrodkin@synopsys.com>
"reset.c" and "cpu.c" have no architecture-specific code at all.
Others are applicable to either ARC CPU.
This change is a preparation to submission of ARCv2 architecture port.
Even though ARCv1 and ARCv2 ISAs are not binary compatible most of
built-in modules still have the same programming model - AUX registers
are mapped in the same addresses and hold the same data (new featues
extend existing ones).
So only low-level assembly code (start-up, interrupt handlers) is left
as CPU(actually ISA)-specific. This significantyl simplifies maintenance
of multiple CPUs/ISAs.
Signed-off-by: Alexey Brodkin <abrodkin@synopsys.com>
Signed-off-by: Igor Guryanov <guryanov@synopsys.com>
Simon Glass